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(57) Abstract 

A machine vision method analyzes a cali- 
bration target of the type having two or more re- 
gions, each having a "imageable characteristic 1 ', 
e.g., a different color, contrast, or brightness, 
from its neighboring region(s). Each region has 
at least two edges - referred to as "adjoining 
edges" - that are linear and that are directed to- 
ward and, optionally meet at, a reference point 
(e.g., the center of the target or some 'other lo- 
cation of interest). The method includes gener- 
ating an image of the target, identifying in the 
image features corresponding to the adjoining 
edges, fitting lines to those edges, and determin- 
ing the orientation and/or position of the target 
from those lines. 
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5 MACHINE VISION CALIBRATION TARGETS AND METHODS 

OF DETERMINING THEIR LOCATION AND ORIENTATION IN AN IMAGE 

Reservation of Copyright 

10 Tne disclosure of this patent document contains material which is subject to 

copyright protection. Tne owner thereof has no objection to facsimile reproduction by 
anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and 
Trademark Office patent file or records, but otherwise reserves all copyright rights 
whatsoever. 

15 • 

Background of the Invention 

The invention pertains to machine vision and, more particularly, to calibration 
targets and methods for determining their location and orientation in an image. 

20 

Machine vision refers to the automated analysis of an image to determine 
characteristics of objects and other features shown in the image. It is often employed in 
automated manufacturing lines, where images of components are analyzed to determine 
placement and alignment prior to assembly. Machine vision is also used for quality 
25 assurance. For example, in the pharmaceutical and food packing industries, images of 

packages are analyzed to insure that product labels, lot numbers, "freshness" dates, and the 
like, are properly positioned and legible. 

In many machine vision applications, it is essential that an object whose image is 
30 to be analyzed include a calibration target. Often a cross-shaped symbol, the target 
facilitates determining the orientation and position of the object with respect to other 
features in the image. It aiso facilitates correlating coordinate positions in the image with 
those in the "real world," e.g., coordinate positions of a motion stage or conveyor belt on 
which the object is placed. A calibration target can also be used to facilitate determining 
35 the position and orientation of the camera with respect to the real world, as well as to 
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facilitate determining the camera and lens parameters such as pixel size and lens 
distortion. 



In addition to cross-shaped marks, the prior an suggests the use of arrays of dots, 
bulls-eyes of concentric circles, and parallel stripes as calibration targets. Many of these 
targets have characteristics thai make difficult finding their centers and orientations. Tnis 
typically results from lack of clarity when the targets and, particularly, their borders are 
imaged. It also results from discrepancies in conventional machine vision techniques used 
to analyze such images. For example, the edges of a cross-shaped target may be 
imprecisely denned in an image, leading a machine vision analysis system to wrongly 
interpret the location of those edges and, hence, to misjudge the mark's center by a fraction 
of a pixel or more. By way of further example, a localized defect in a camera lens may 
cause a circular calibration mark to appear as an oval, thereby, causing the system to 
misjudge the image's true aspect ratio. 



In addition to the foregoing, many of the prior an calibration targets are useful 
only at a limited range of magnifications. Parallel stripes, for example, do not provide 
sufficient calibration information unless many of them appear in an image. To 
accommodate this, a machine vision system must utilize lower magnification. However, 
20 as the magnification decreases, so does the ability of the machine vision equipment to 
distinguish between individual stripes. Similar drawbacks limit the usefulness of the other 
pnor an calibration targets for use in all but a narrow range of magnifications. 

Though the an suggests the use of checkerboard patterns as alignment marks, the 
25 manner in which images of those marks are analyzed by conventional machine systems 
also limits their utility to a limited range of magnifications. Particularly, prior art systems 
obtain alignment information from checkerboard marks by identifying and checking their 
corners, e.g., the eight black (or white) corners in a black-and-white image. By relying on 
corners, the systems necessitate that images show entire checkerboards, yet, with 
30 sufficient resolution to insure accurate detection and analysis. 
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An object of this invention is to provide an improved calibration targets and 
methods for machine vision analysis thereof 

A related object is to provide calibration targets and analysis methods reliable at a 
5 wide range of magnifications. 

A further object is to provide such methods as can be readily implemented on 
conventional digital data processors or other conventional machine vision analysis 
equipment 

10 Yet still another object of the invention is to provide such methods that can rapidly 

analyze images of calibration target without undue consumption of resources. 
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Summarv of the Invention 



The foregoing objects are among those attained by the invention, which provides 
in one aspect a machine vision method for analysis of a calibration target of the type 
having two or more regions, each having a different "imageable characteristic" (e.g. 7 a 
different color, contrast, or brightness) from its neighboring region(s). Each region has at 
least two edges p referred to as "adjoining edges" - that are linear and that are directed 
toward and, optionally meet at, a reference point (e.g., the center of the target or some 
other location of interest). The method includes generating an image of the target, 
identifying in the image features corresponding to the adjoining edges, and determining 
the orientation and/or position of the target from those edges. 

In another aspect, the invention provides a method as described above for 
analyzing a target of the type that includes four regions, where the adjoining edges of 
each region are perpendicular to one" another, and in which each region in the target has a 
different imageable characteristic from its edge-wise neighbor. The edges of those regions 
can meet, for example, atjhe center of the target, as in the case of a four-square 
checkerboard. 

In yet another aspect, the invention provides a method as described above for 
determining an orientation of the target as a function of the angle of the edges identified in 
the image and for determining the location of the reference point as an intersection of lines 
fitted to those edges. In regard to the former, the invention provides a method of 
determining the orientation of a target in an image by applying a Sobel edge to ol to the 
image to generate a Sobel angle image, and by generating a angle histogram from that 
angle image. In an alternate embodiment, the orientation is determined by applying a 
Hough line tool to the image and determining the predominant angie of the edges 
identified by that tool. 

In regard to the location of the reference point, one aspect of the invention calls for 
locating the adjoining edges by applying a caliper vision tool to the image, beginning at an 
approximate location of the reference point. That approximate location of the reference 
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point can itself be determined by applying a Hough line vision tool to the hnage in order 
to find lines approximating the adjoining edges and by determining an intersection of 
those lines. Alternatively, the approximate location of the reference point can be 
determined by performing a binary or grey scale correlation to find where a template 
representing the edges most closely matches the image. 

In another alternate embodiment the approximate location of the reference point is 
determined by applying a projection vision tool to the image along each of the axes with 
which the adjoining edges align. A first difference operator vision tool and a peak 
detector vision tool are applied to the output of the projection tool (i.e., to the projection) 
in order to find the approximate location of the edges. 

The invention has wide application in industry and research applications. It 
facilitates the calibration of images by permitting accurate determination of target location 
and orientation, regardless of magnification. Thus, for example, an object bearing a tareet 
can be imaged by multiple cameras during the assembly process, with accurate 
determinations of location and orientation made from each such image. 

These and other aspects of the invention are evident in the drawings and in the 
description that follows. 
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A more complete understanding of the invention may be attained by reference to 
the drawings, in which; 

5 

Figure 1 A - 1C depict calibration targets according to the invention; 

Figure ID depicts the eSect of rotation on the target depicted Figure IB; 

10 Figure 2 depicts an object according to the invention incorporating a calibration 

target of the type depicted in Figure IB; 

Figure 3 depicts a machine vision system according to the invention for 
determining the reference point and orientation of a calibration target; 

15 

Figures 4 and 5 depict a method according to the invention for interpreting an 
image of a calibration target to determine a reference point and orientation thereof; and 

Figure 6 illustrates the magnification invariance of a target according to the 
20 invention. 
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Detailed Description of the Illustrated Embodiment 

Figures 1A - 1C depict calibration targets according to the invention. Referring to 
Figure 1 A, there is shown a target 10 according to the invention having three regions 12, 
5 14, 16. Each region is bounded by at least two linear edges that are oriented toward a 
reference location or reference point 18 on the target. Thus, for example, region 12 is 
bounded by edges 20, 24; region 14 is bounded by edges 20, 22; and region 16 is bounded 
by edges 22, 24. As evident in the drawings, the edges are shared by adjoining regions 
and, hence, are referred to below as "adjoining edges." Thus, region 12 shares edge 20 
10 with region 14; region 14 shares edge 22 with region 16; and region 16 shares edge 24 
with region 12. In the illustration, the reference point 14 is at the center of target 10, 
though, those skilled in the an will appreciate that the reference point can be positioned 
elsewhere. 

15 Each of the regions has a different imageable characteristic from its neighboring 

regions. As used herein, an "imageabie characteristic' 1 is a characteristic of a region as 
imaged by a machine vision system (e.g., of the type shown in Figure 3) and, particularly, 
as imaged by an image capture device used by such a system. For example, in the 
illustration, region 12 has the characteristic of being colored black; region 14. white; and 

20 region 16, gray. In addition to coicr, imageable characteristics useful with conventional 
machine vision systems - which typically utilize image capture devices operational in 
visual spectrum — include contrast, brightness, and stippling. 

Those skilled ;in the art will appreciate that any other characteristics by which a 
25 region may be identified and distinguished in an image are suitable for practice of the 
invention. Thus, for example, for a machine vision system that utilizes a temperature- 
sensitive (or infrared) image capture device, an imageable characteristic is temperature. 
By way of further example, for a machine vision system that utilizes a nuclear decay 
radiation-sensitive image capture device, an imageable characteristic is emitted radiation 
30 intensity or frequency. 
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As shown in the illustration the adjoining edges 20, 22, 24 comprise straight linear 
segments. Tnose edges are implicitly defined as the borders between regions that, 
themselves, have different imageable characteristics. Thus, for example, edae 20 is a 
straight linear segment denned by the border between black region 12 and white region 
14. Likewise, edge 24 is^defined by the border between black region 12 and gray region 
16. Further, edge 22 is defined by the border between white region 14 and grey region 16. 

Figure IB depicts a calibration target 30 according to the invention having four 
rectangular (and, more particularly, square) regions 32, 34, 36, 38. As above, each region 
is bounded by at least two linear edges that are oriented toward a reference point 40 at the 
center of the target. Thus, for example, region 32 is bounded by edges 42, 44; region 34 is 
bounded by edges 42, 46; and so forth. As above, these edges are shared by adjoining 
regions. Thus, region 32 shares edge 42 with region 34, and so forth. Each region in 
target 30 has a different imageable characteristic from its edge-wise neighbor. Hence, 
regions 32 and 36 are white, while their edge-wise adjoining neighbors 34, 38 are black. 

Figure 1C depicts a calibration target 50 according to the invention having five 
regions 52, 54, 56, 58, 60, each having two linear edges directed toward a reference point 
62. The adjoining regions are of differing contrast, thereby, denning edges at their 
common borders, as illustrated. Although the edges separating the regions 52 - 60 of 
target 50 are directed toward the reference point 62, they do not meet at that location. As 
evident in Figure IC no marker or other element imageable characteristic is provided at 
reference point 62. 

Those skilled in the art will appreciate that, in addition to the calibration targets 
shown in Figures 1A - IC, targets with still more regions (or as few as two regions) and 
shapes, otherwise in accord with the teachings hereof fall within the scope of the 
invention. Moreover, it will be appreciated that targets may be of any size and that their 
regions need not be of uniform size. Still further, it will be appreciated that the outer 
borders of the targets need not be linear and may, indeed, take on any shape. 
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Figure 2 depicts an object according to the invention for use in machine vision 
imaging, detection, and/or manipulation having a calibration target according to the 
invention coupled thereto. In the illustration, the object is an integrated circuit chip 70 
having coupled to the casing thereof a calibration target 72 of the type shown in Figure 
IB. Other targets according to the invention, of course, can likewise be coupled to the 
object 70. The targets can be coupled to the object by any known means. For example, 
they can be molded onto, etched into, or printed on the surfaceof the object. By way of 
further example, decals embodying the targets can be glued, screwed or otherwise affixed 
to the object. Moreover, by way of still further example, calibration plates incorporating 
the targets can be placed on the object and held in place by friction. In addition to 
integrated circuit chips, the object can include any other objects to which a target can be 
coupled, such as printed circuit boards, electrical components, mechanical parts, 
containers, botties, automotive pans, paper goods, etc. 

Figure 3 depicts a machine vision system SO according to the invention for 
determining the reference point and orientation of an object 82 having coupled thereto a 
calibration target 84 according to the invention and, particularly, a four-region target of 
the type shown in Figure IB. The system 80 includes an image capture device 86 that 
generates an image of a scene including object 82. Although the device may be 
responsive to the visuai spectrum, e.g., a conventional video camera or scanner, it may 
also be responsive to emissions (or reflections) in other spectra, e.g., infrared, gamma-ray, 
etc. Digital image data (or pixels) generated by the capturing device 86 represent, in the 
conventional manner, the image intensity (e.g., contrast, color, brightness) of each point in 
the field of view of the capturing device. 

That digital image data is transmitted from capturing device 86 via a 
communic2;;ons path 88 to an image analysis system 90. This can be a conventional 
digital data processor, or a vision processing system of the type commercially available 
from the assignee hereof, Cognex Corporation, as programmed in accord with the 
teachings hereof to determine the reference point and orientation of a target image. The 
image analysis system 90 may have one or more central processing units 92, main memory 

9 
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94, input-output system 96, and disk drive (or other mass storage device) 98, all ot tfte 
conventional type. 

The system 90 and, more particularly, central processing unit 92, is configured by 
programming instructions according to teachings hereof for operation as illustrated in 
Figure 4 and described beiow. Those skilled in the art will appreciate that, in addition to 
implementation on a programmable digital data processor, the methods and apparatus 
taught herein can be implemented in special purpose hardware. 

Referring to Figure 4 there is shown a machine methodology according to the 
invention for interpreting an image of a target 84 to determine its reference point and 
orientation. The discussion that follows is particularly directed to identifying a four- 
region target of the type shown in Figure IB. Those skilled in the an will appreciate that 
these teachings can be readiiy applied to finding targets according to the invention, as well 
as to other targets having detectable linear edges that are oriented toward a reference 
location or reference point on the target, e.g., a prior art cross-shaped target. For 
convenience, in the discussion that follows, such linear edges are referred to as "adjoining 
edges," regardless of whether they are from calibration targets according to the invention 
or from prior art calibration targets. 

In step 100, an image of the target 84 (or of the target 84 and object 82) is 
generated, e.g., using image capture device 86, and input for machine vision analysis as 
discussed beiow. The image can be generated real time, retrieved from a storage device 
(such as storage device 98), or received from any other source. 

In steps 102 - 108, the method estimates the orientation of the target in the image 
using any of many alternative strategies. For example, as shown in step 102, the method 
determines the orientation by applying a conventional Hough line vision tool that finds the 
angle of edges discernabie in the image. In instances where the target occupies the entire 
image, those lines will necessarily correspond to the adjoining edges. Where, on the other 
hand, the target occupies only a portion of the image, extraneous edges (e.g., from other 
targets) may be evident in the output of that tool. Although those extraneous edges can 

10 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 m . PCT/US97/18268 

"'"^ generally be ignored, in instances' where they skew the results, the image can be 

windowed so that the Hough vision tool is only applied to that portion that contains the 
target. Once the angles of the lines has been determined by the Hough line tool, the 
orientation of the image is determined from the predominant ones of those angles. 

5 Alternatively, the angle of the image* can be determined by taking a histogram of the 
angles. 

The Hough vision tool used in step 102 may be of the conventional type known 
and commercially available for finding the angle of lines in image. A preferred such tool 
10 is the Cognex Line Finder, commercially available from the Assignee hereof, Cognex 
Corporation. 

An alternative to using a Hough vision tool is shown in step 106. There, the 
illustrated method determines the orientation of the target by applying a Sobei edge tool to 
15 the image to Snd the adjoining edges. ^Particularly, that tool generates a Sobei angle 
image that reveals the direction of edges in the image. As above, where the target 
occupies the entire image, the adjoining edges will be the only ones discerned by the 
Sobei edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 

20 

The Sobei edge tool may be of the conventional type known and commercially 
available for nnding lines in image. A preferred such tool is the Cognex Edge Detection 
tool, commercially available from the Assignee hereof, Cognex Corporation. 

25 Once the Sobei angle image is generated, in step 106, the orientation of the target 

in the image is determined by generating a histogram of the edge angle information; see, 
step 108. From that histogram, the target orientation can be determined by taking a one- 
dimensional correlation of that histogram with respect to a template histogram of a target 
oriented at 0°. Where a Sobei magnitude image is generated, in addition to the Sobei 

30 angle image, such a histogram can be generated by counting the number of edges greater 
then a threshold length at each orientation. 
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As a still further alternative to applying a Hough vision tooi or Sobei edse tool, the 
method contemplates obtaining the angle of orientation of the target from the user (or 
operator). To this end, the user may enter angle orientation information via a keyboard or 
other input device coupled with digital data processor 90. 

In steps 1 10 - 1 18, the method detennines the location, Le., coordinates, of the 
target reference point in the image. Particularly, in step 1 10, the method can apply a 
Hough vision tooi, as described above, to 2nd the angle of lines discernable in the image. 
A conventional Hough vision tool determines, in addition to the angle of lines in an image, 
the distance of each line, e.g., from a central pixel. As above, where the target occupies 
the entire image, those lines will be the oniy ones discernable by the Sobe! edge tool 
image. Where, on the other hand, the target occupies oniy a portion of the image, any 
extraneous edges can be ienored or windowed out. 



As above, the Hough vision tooi used in step 104 may be of the conventional type 
known and commercially available for finding the angle and position of lines in image. 
Once again, a preferred such tool is the Cognex Line Finder, commercially available from 
the Assignee hereof, Cognex Corporation. Those skilled in the an will, of course, 
appreciate that steps 102 and 1 10 can be combined, such that a single application of the 
Hough vision tool provides sufficient information from which to determine both the 
orientation of the target in the image and its reference point. 



As an alternative to using a Hough vision tool, the method can apply a projection 
vision tool to the image in order to find the position of the lines discernable in the image; 
see, step 1 12. The projection tool, which maps the two-dimensional image of the target 
into a one-dimensional image, is applied along the axes denned by the edges in the image. 
As those skilled in the art will appreciate, the location of the edges can be discerned from 
by finding the peaks in the first derivatives of each of those projections. As above, where 
the target occupies the entire image, those lines will be the only lines discernable by the 
Sobel edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 
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The projection vision tool used in step 1 12 may be of the conventional type known 
and commercially available for mapping a two-dimensional image of the target into a one- 
dimensional image. A preferred such tool is that provided with the Cognex Caliper tool 
commercially available from the Assignee hereof Cognex Corporation. 

5 

In step 1 14, the method uses the information generated in steps 1 10 and 1 12 to 
compute the location of the reference point, particularly, as the intersection of the lines 
found in those steps 1 10 and 1 12. 

10 As an alternative to using the Hough vision tool and the projection tool, the 

method can apply determine the location of the reference point by performing a binary or 
grey scale correlation on the image; see step 116. To this end, the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross-shaped pattern in the case of a target of the type shown in Figure IB. The use of 

15 correlation vision tools for this purpose is well known in the art. The template for such an 
operation is preferably generated artificially, although it can be generated from prior 
images of similar targets. 

As still another alternate to the Hough vision tool and the projection tool, the 
20 method can apply a grey-scale image registration using the sum of absolute differences 
metric between the image and a template; see step 118. To this end. the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross- shaped pattern in the case of a target of the type shown in Figure IB. The template 
' for such ah operation is preferably generated artificially, although it can be generated from 
25 prior images of similar targets. A preferred grey-scale image registration tool is disclosed 
in United States Patent No. 5,548,326, the teachings of which are incorporated herein by 
reference. 

Although steps 1 10 - 118 can be used to determine the approximate location of the 
30 reference point of the target in the image, the method utilizes optional steps 120 and 122 
to refine that estimate. These two steps are invoked one or more times (if at all) in order 
make that refinement. 

13 
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In step 120, the method applies a conventional caliper vision tool to tmd points in 
the image that define the adjoining edges of the regions. On the first invocation of step 
120, the method applies calipers along fifty points along each edge (though those skilled 
in the an will appreciate that other numbers of points can be used), beginning with points 
5 closest to the estimate of the reference point, as discerned in steps 110-118. The calipers 
are preferably applied a small distance away from the actual estimate of the reference 
point to avoid skewing the analysis due to possible misprinting of the target at that point, a 
missing pattern at that point (e.g., Figure 1C), or a too-high spatial frequency at that point 
(e.g., Figure IB). In step 120, the method then fits a line to the points found along each 
10 edge by the caliper tool, preferably, using a conventional least squares technique. 

In step 122, the method computes a refined location of the reference point as the 
intersection of the iines identified in step 120. Although conventional geometric 
calculations can be performed for this purpose, preferably, the reference point is computed 

15 using conventional least squares techniques. Preferably, when examining the image of a 
symmetric calibration target of the type shown in Figure IB, the method utilizes the same 
number of points on either side of (and closest to) the reference point for purposes of 
fitting each line. This minimizes the bias otherwise introduced by a conventional edge 
detection technique in finding edges that are defined only by dark-to-light (or light-to- 

20 dark) transitions. 



Calibration targets of the type shown in Figure IB are advantageously processed 
by a method according to the invention insofar as they further minimize bias otherwise 
introduced by a conventional edge detection techniques. In this regard, it will be 
appreciated that such bias is reduced by the fact that "opposing" adjoining edges (i.e., 
edges that oppose one another across the reference point) define straight linear segments 
that change polarity across the reference point. That is, those segments are denned by 
regions that transition -- preferably, equally in magnitude - from light-to-dark one one 
side of the reference point, and from dark-to-light on the other side. This is true for all 
"symmetric" calibration targets according to the invention, i.e., targets in which opposing 
edges define straight linear segments that are opposite polarity on either side of the 
reference point. 
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On the second and subsequent invocations of step 120, the method specifically 
applies the caliper tool at points along the lines found in the previous iteration. 
Preferably, these points are at every pixel in the image that lies along the line, though, 
those skilled in the an will appreciate that few points can be used. The calipers can be 
5 applied orthogonal to the lines, though, in the preferred embodiment, they are applied 
along the grid defined by the image pixels. The caliper range decreases with each 
subsequent invocation of step 120. Tne method continues applying the calipers, beginning 
at the estimated center point until one of the four following situations occurs: no edge is 
found by the caliper applied at the sample point; more than one edge is found by the 

10 caliper and the highest scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the CONFUSIONJIHRESHOLD); the distance 
between a computed edge point and the nominal line (computed from the previous 
invocation of step 120) is larger than a threshold (which threshold decreases with each 
subsequent invocation of step 120); or, the caliper extends outside of the image. As 

15 above, in step 120, the method then Sis a line to the points found along each edge by the 
caliper tool, preferably, using a conventional least squares technique. 

In the second and subseaueni invocations of step 122, the method computes a 
refined location of the reference point as the intersection of the lines identified in step 120. 
20 As above, although conventional geometric calculations can be performed for this 
purpose, preferably, the reference point is computed using conventional least squares 
techniques. 

Referring to Figure 5, there is also shown a machine methodology according to the 
25 invention for interpreting an image of a target 84 to determine its reference point and 
orientation. Particularly, in step 150, the method calls for generating an image of a target 
84 and, particularly, of a target according to the invention having two or more regions, 
each region being defined by at least two linear edges that are directed toward a reference 
point, and having at least one of the regions having a different imageable characteristic 
30 from an adjacent region. Step 152 can be effected in the manner described in connection 
with step 1 00 of Figure 4, or equivalents thereof. 
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In step 152, the method analyzes the image to generate an estimate of an 
orientation of the target in the image. Step 152 can be effected in the manner described in 
connection with steps 102- 108 of Figure 4, or equivalents thereof. 

In step 154, the method analyzes the image to generate estimate of a location of the 
target's reference point. Step 154 can be effected in the^manner described in connection 
with steps 1 10 - 1 18 of Figure 4, or equivalents thereof. r 

In step 156. the method analyzes the image to refine its estimates of the location of 
the reference point in the image and the orientation of the target in the image. Step 156 
can be effected in the manner described in connection with steps 120 - 122 of Figure 4, or 
equivalents thereof. 



Calibration target and methods for analysis according to the invention are 
15 advantageous over prior art targets and methods insofar as they are magnification 

invariant. By analyzing the adjoining edges of targets, merhods according to the invention 
insure reliance on features (to wit, regions) that retain the same imageable appearance 
regardless of magnification. This is in contrast to prior art targets and methods, which rely 
on individual lines (or dots) to define calibrating features. -As noted above, the imaging 
20 appearances of such lines and dots change with varying degrees of magnification. Even 
the prior an methods that analyze checkerboard targets rely on analysis of corners, which 
are not magnification invariant. 

The magnification invarianee of targets and methods according to the present 
25 invention is illustrated in Figures 6A - 6C. Referring to Figure 6A, there is shown an 
imaging setup wherein camera 200 images a target 202 (on object 204) from a height x. 
An image generated by camera 200 is displayed on monitor 206 of workstation 208. 
Referring to Figure 6B, there is shown an identical imaging setup, except insofar as a 
camera (of identical magnification) images a target (of identical size) from a greater 
30 height, x 1 . Likewise, Figure 6C shows an identical imaging setup, except insofar as a 
camera (again, of identical magnification) images a target (again, of identical size) from a 
still greater height, x'\ Comparing monitor images depicted in Figures 6A - 6C, it is seen 
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that the images of the targets, generated by the cameras and displayed on the monitors, are 
identical in appearance regardless of the relative heights (x, x', and x") of the camera. This 
magnfication invariance results from the feet that the target has the same appearance 
regardless of height (or equivalently, magnification), within specified limits thereof. 
5 Those skilled in the art will appreciate that those limits are greater than the analagous 
limits for prior art targets (e.g., cross hairs, parallel stripes, etc.). 

Still another advantage of calibration targets and methods according to the 
invention is that they permits angular orientation to be determined throughout a full 360° 

10 range. With reference to Figure ID, for example, the relative positions of the regions can 
be used to determine the overall orientation of the target, Le., whether it is rotated 0°, 90°, 
180°, or 270°. This information can be combined with a determination of relative 
orientation made by analysis of the adjoining edges as discussed above to determine the 
precise position of the target. 

15 ' ' ...... 
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APPENDIX I '• 



Patent Application for 



MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AMD ORIENTATION 



Vision Tool Description 



THE FOLLOWING APPENDIX IS NOT BELIEVED TO BE NECESSARY FOR 
ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED 
CLAIMED IN THE ACCOMPANYING APPLICATION. 
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Edge Detection Too! 

This cnacter essences ma Ecca Detection :coi. which r.ciuces ecge cstscticn anc Desk 
detection. 

E$ga detection takes an input 'macs anc crcducas rwc cutcu: images: an image oi the 
edge magnitude of eacn input pixel ana an image of me ecge angle or aacn input pixei. 
When combined wim ether Cognex software toois. rne infcrmaccn produced by edge 
detection can be used to locate ocjects wnmn an image. Edge pixei information can oe 
used :c cetect the rotation of an obiect or cefects sucn as scratches, oracles, or oarticias. 
You can ajso gauge now snarpry an image is fecusse witn ecge sixei information. 

Peak dotocticn is uuiut in any aopiica&on tn whicn xnowteogo cf mo iocat maximum voJuee 
;n a two-dimensional soaca & useful. PeaK detection takes an input image ana crccuces 
an ourcut image containing only tnose oixeis in the incut imacs with nigner values man 
neighboring oixeis. A tycicai incur image to ceak detection is an ©cga magnitude image: 
me cuicut image contains cmy me mgnest magnnuae ecce oixeis. The eace cetection 
function can ccocnaily use oeaK deiectJcn to postcrocass me ecge cetscnen results so 
mat only the strongest ecges remain. 

This cnacter nas rane sections as follows: 

An Overview of Edge Detection describes the goais of edge detection, defines an edge 
pixel, and describes now the Edge Detection toot finds edge puas. This section also 
. explains me two crcpenies of a pixel that are calculated oy the Edge Detection tool, ecge 
magnitude and ancle, and ends with a samcle aoolication. 

Using Edge Detection descrices, in general terms, the mterfaca to me Edge Detection tool. 
Tms section contains a aiscusston of tne compression laoies mat affect me amount of 
memory used Dy tne ecge detector. 

Edge Detection Enumerations and Data Structures oescrioes the aata structures and 
enumerations mat me edge catacticn functions use. Types ana cata structures mat 
support oeak cetect cn are ctscussad in Peak Detection Enumeration Types and Data 
Structures befew. 
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\ Edge Datacuon Functions describes the functions that mplement edge detection. 

- Maximizing the Performance of VC2 V&ori Coorpcassor lists tne edge detection parameter 
settings that you should avoid^t youwant to maximize the oerformance of tne VC2 vision 
coprocessor during edge detection. 



: Overviaw of Peak Detection defines peak pixel and gives examoles of seme acplicanonc 
of pook doioction. a: j% : 

Using Peak Detection describes me^ntertace to peak detection. 

Peak Detection Enumeration Types and Data Strvctw as describes the peak detection 
enumerations and aata structures.'Ji^i^ j£i ... 

Peak Detection Fttncricns doscnoas W.oaak da taction functions. 
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Edge Detection Tool 



Some Useful Definitions 



edge phtei 

edge detector 
compr eeeton table* 



Sobei operators 

horizontal edge 
component 

▼irttcai *dgo component 
edge magnitude 
edge angle 



pak pixel 

"peek detection 

d-point neighborhood 

4-porm neighborhood 
2 -point neighborhood 



Pixel that nas a different vaiue from one or mere of its 
etgm neignconng ptxeis. 

Operator thot fines acge pixels in an imago. 

Two tables -fiat are created during edge detection 
initiaiizaDcn: the ecge compression tacie ana Tie 
magntude compression table. These tactes are used by 
tne Edge Detection toot to map aaxa :mo a arr.aJier range 
of values. 

Two 3x3 operators mat tne Ecge Detection :cci uses to 
locate edga pixais in an image. 

Value returned by tne hcrizcntai Sccei coeratcr wnen it is 
appiiea to a pixel in an image. 

Value returned by the vertical Scbai cserator wrten it :s 
applied to a pixel in an image. 

value mat increases as the difference tn grey levels 
between neignoormg pixels increases. 

Orientation of an ecge. with respect ro tne x*axis of me 
image, zi an edge pixel. The sdga Oetscticr, tool CGfines 
tne angle of a pixel as arc:an(w?i). wnere /is the vertical 
eage component and nia me nonrontai ecge component 
of the pixel. 

Pixei with a vaiue greater than or ecual to seme or ail of 
Us neighboring pixels' vaiues. 

Finding seme or all peak pixels in an 'mage. 

Pixels mat are horizontally, vertically, and diagonally 
adjacsnt to a pixei. 

Pixels that are horizontally and venicalry adjacent to a 
pixel. 

Neighbors cf a pixei aicng a single axis. Psa* detection 
supports a horizontal and a verrical 2-coint neighbornccd 
operator. 
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symmetric peak 
asymmetric peak 

•iMxea peak 
aingte-exle peak 
plateau 



Peak wnose ptxet value is greater man or equaJ to the 
values of each of its neignoors. 

Peak wnosa pixel value is greater than tfte values cf its left 
and lower neighoors and graator than or aqua) to the 
values of Its rtgnt and upper netgnocrs. 

Pixel that satisfies tne peak definition (symmetnc or 
asymmetric) along all axes in its neighborhood. 

Pixel that satisfies 'me oeak definition (symmetric or 
aaymmetno) along at toast one axia tn its neighborhood. 

In oeak detection, a recicn of contiguous oixels of tne 
same value. 
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An Overview of Edge Detection 



The Edge Detection tool fines ecge Duels ;n an image. This overview of ecge detection 
contains tho fcllowing daccnoiions: 

• Definition of me problem of ecge detection 

• Overview of tne tasks mat ?ie Ecge Detection tcct performs 

• Definition cf ecge pixel 

. DescrioDcn cf the Scoel coeratc:. wmch is Ltssc bv Tie Edce Deiecr.cn tccl :o locate 
ecge pixels 

. Definitions of trio angle and magnituce cf an eogo pixel, ana deeeriotione of ;ho rwo 
images produced oy tne Ecge Detection tcct: the magnitude image and me angle 
image 

• Description of ecge detection preprocessing 

• Description of edge detection postprocessing 

• Description cf a sampie apoiicatlcn mat uaea bcth the magnituce and Tie angie 
images to create an angie "signature' of a shspe 



Edge Detection 

The Edge Detection tcol locates edges and determines each edge's angle and magnituce. 

An edge occurs wherever adjacent areas of an image have affl erent grey values. Usng this 
definition. :ns triangie in Figae 6S has a single edoe. 




Figure 66. A figure with an edge 
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The definition of edge can be runner refined as ma directed border between cyey areas ot 
an image, where dtrecnon ts defined as a vector normal to the edge. Using this definition, 

the triangle in Figua S7 has :nrea airectional adges, raprosontad by tn« thraa vectors. Tho 
angte ot the edge is tne couniercxocwise angle of tne vector normal to the eage, with 
resoect to the horizontal axis of tne ;mace. See the section Angle Image on oage 181 for a 
discussion of edge angle. 




Figura 67. vec:crs ;r.cicavng no ecge angles of a triangle 

Along with an angte, an scga nas a magnttudQ, wnicn reflect the amount of Tie differanca 
between grey ievets on eitner sice of tr^e acge. As me difference in grey levels increases, 
the magnitude increases. See the section The Magnitude Image on oage 178 for a 
discussion of edge magnitude. 

The triangle in Figure 58 has me 3am e :r.ree edges as tne trr angle in Figure 57; however, 
these edges are of lower macnituce. This sower magnitude is illusrrated by me sncrteneo 
lengtn of tne tnree direction vectors. 




Figure 68. A mangle with low*magnituce edges 
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Edges are actually located pixel by sued. Figure 69 contains a magnified view of a portion 
of an edge in an Image. Each call m the figure represents the grey level or a single pixel. 
:Notice that, on this Tniaoscooic" level, there are many edges between oixels in whicn me 
grey levels on either sice cf Tie edge differ by a few percentage points. 
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figure $Q. Grey levels in a msgnifiad poftion cf ecge 

The Edge Detection tool locates ecges in an image by identifying each pixel that has a 
different value from one of its neignooring oixeis. It calculates the ancie arc magnitude 
eacn edge pixel, it also orovides a means of classifying ecges. so mat icw-magnituce 
edges are not reported. 
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Edge Pixels 



A pixel is an &ag& oixoltt it haa a differ ant value frcm at loaat ono cf its oigm neighboring 
pixels ana is nor on me scrcer of an image. !n Figure 70. tha fcur snaaea pixels are acge 
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Figure 70. The shaaed pixeis are ecga pixels 

Figure 7t contains a grid representing grey levete in an image wim tie hignest magnitude 
edge pixels shaded. Notice mat borcsr pixeis (pixeis aicng the eage cf the image) are not 
edge pixels. 



0 0 0 


0 0 


0 


0 


0 ) 0 


0 0 


0 0 0 


0 | 0 


0 


0 0 




3 


0 0 


0 


0 0 


0 










4 






0 


0 


0 


0 | 0 




4 


4 


4 


4 




0 


0 


0 


'TT'^fcf ' 


4 


4 


4 


4 


4 


4 


0 tPSSI 


4 | 4 


4 


4 


4 


4 


4 


4 


3 j 4 |% 


4 | 4 


4 


4 


4 


4 


4 


4 


4 4 


4 


4 4 


4 


4 4 


4 


4 


4 


4 


4 


4 


4 4 


4 


4 4 


4 


4 


4 



figure 71. Hignest magnitude aage pixete in an imago 
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The Edge Detection Operators 

The Edge Detection tool find* ocge pixels using the Sob* 3*3 neighbornood operator*. 
During edge detection, the Sccei operators are applied to each pixel in an input image. 
This operation orccuces two values for each pixel. One value r Qoresonis the vBrtcai edge 
component the oixsi; the other value represents the nonzcnreJ edge component tor the 
pixel. These two values are then used to compute me edge magnitude and edge angle of 
the pixel. 

Sobei edge detection uses two 3x3 neighocmocd ooeratora to locate edge pixeta In an 
image The horizontal Sobei operator detects the norizcntal (x) edge component for a ptxei. 
Tne vertical Sccei operator Detects tne vertical (y) eoge component tor a oixei. Figure 72 
snows the Sobei operators. 
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Figure 72. Sccei operators 

The Sobs! operator works only on pixels with eight neighooring pixels. Wnen the Sobei 
operator is apolied to a pixel on the border of an image, tne result is defined to be 0 
because there are not enougn neighbcr.ng pixels to calculate an edge value. 

The horizontal ana vertical Sobei operators are 3x3 linear operators. These operators 
compute a value for a pixel (x.y) in the following way: 

• The grey level of tne otxei (x.y) is muitiolied by the value in the center of the 3x3 linear 
operator. 

• The grey level of the neighboring pixel (x-i.y-D is multiplied by the first value in the 
. top row of . the 3x3 linear operator. '/-..>"'• 

"• The grey level of me neighboring pixel U/-7) is multiplied by the second value in the 
top row of the 3x3 linear operator. 

• The continues untd tne produot of eecn peir of valuee is calculated. 

. These products are tnen summed to produce the ouffiul For the Sobei operator the 
vaiue ts inter pretea as Horizontal or vertical eags component or me oixel. 



27 



SUBSTITUTE SHEET {RULE 26) 



WO 98/18117 



PCT/US97/18268 



4 Edge Detection Tool 



If tfte result cf aopiying at leas; cr.e of the Socei cDeratcrs re 3 Dixei is nonzero, that pixel 
is an edge pixel. 



-1 


0 


1 


-2 


0 


2 


-1 


0 


1 



Scbei oceraicr 



1 


1 


2 


1 




3 




2 | 3 


3 



image pixels 



f/gure 73. Acctying the Sotei cceratcr 

In figure 73, the nonzcntal Sobei oceraicr is acolled to the snaced o:xd in me :mage. The 
reauitant edge oixei value is 6. wnicn is calculates as follows: 

-1-I*0-UI-2t-2-U0.2t2-3-p-.1-2*0-3*1-3 = 6 

A/ore: Tne range cf cixel values in Figure 73 and in mosi examples in mts ccc^nent is rnucn 
smaller than the rypicai range of cixel values in an image. This is dens tc simplify :ne 
examples. 

Figure 74 shews all the vertical ar.a horizontal aage component values for a small image. 
The uocer grid is the image; :ha vaiues represent the grey feveis of each pixel. The gna to 
tne icwer !en cf tne irnaca contains nonzontal ecge ccmocnent values tor the :msee. £ach 
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call in the grid contains we result of applying the Sobel hcri2cntai operator :o me 
corresponding input image pixel. The grid to the lower right contains the vertical ecge 
comoonents tor me image. It is comcuted using me Socal verocai operator. 
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Horizontal ecge component vaiues 
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Vertical edge comocnent values 



Rgure 74. Horizontal ana vertical ecge component vaJues computed 
with the Sobei operators 

The horizontal and vertical eags component values computed with the Scfcel operator are 
not returned by the edge detection funcacns. They are usea to compute me edge 
magntuce ana ecge angie or eacn pixel in me image. 
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The Output Images 



£dgo magnitudes aro ctcrao in an outDut imago That snows tna edg« magnitude of each 
corresponding pixel in ihe input image. Edge angies are stored in anotnef cutout image 
mat shows the edce 3ncle cf each oixsi. Ycu can chocso wnicn cutout images to craata: 
magnitude, angie, or both. 

These two images are described in this section. 

Th« Magnitude Image 

Tne ecge magnrtuae for a given Dixel is s function cf the horizontal and vemcal edge 
ccmoonents cf me pixel. If jt is me ncrizcnrai edge component value and yis the vertical 
ecge component value ror seme oixet. man tne ecgemagnituce. m. tor mar pixel is aarinea 
as foiicws: 



This fcrmuia suggests a geometric interpretation cf tne oata: if x\% the horizontal edge 
component ana yis mo vertical ecge component. Tie magnitude is a vector from me origin 
of a two-dimensional coordinate system to me point (x.y). Figure 75 shews the geometric 
r eletionehip between ine ecge magnitude ana the rwo edge oomoonents. 



Note: For jmclementatjon-soecific t easons, the Edge Detection tool uses me formula 
described above to compute edge magnitude ana then scales me magnitude upward 
by approximately »6%. 




Vertical edge y ~ 
component * 




Eage magnitude 



* Horizontal edge 
component 



Figure 75. Geometric representation of edge magnitude 
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Ycu can cnoose me numoer of bits mat can be used to express magnitude. Because 
magnitude values, computed with vertical and horizontal edge component values, can 
exceed this number cf bits, magnitude values are ccmoressed to an integer value mat can 
oe expressed in trie number cf bits ycu have specified. You can control now magnitude 
values are compressed: you can choose a metnod succiied by the Eggs Detection tool, 
such as a logarithmic or linear map. or you can supply a magnitude compression map of 
your* own. For descriptions of the macs supplied by the £dge Detection tool and for 
information on using your own map, see me section Two Compression Tabiason page 192. 

The sae and deoih of the magnitude image are the same as that of tne inout imago. Since 
au border pixels have horizontal and vertical edge pixel values cf 0. the border cf me 
magnitude image ccntaira all zercs. Ycu can diaelay an edge magnttuce image: higner 
magnitude edges are brignrer. 

Octicnaily. edge detection can return tne numoer cf edge cixeis in me incut image ana trie 
sum of all edge magnitude values. By civiaing tne sum of ail ecge magnitude vaiues oy tne 
size cf the image, ycu can compute tne average edge magnitude of an image, [f ycu have 
two images of the same scene, you can determine whicn image is in snarper focus by 
comparing average edge magnitudes of the two images: the image witn tne higher average 
magnitude is in snarper focus. 
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Figure 76 contains an image, rhe imaged horizontal and vertical edge ccmconents 
commuted with Sobei operators, and me output image containing the edge magnitude 
values fer each oixel. The edge onceis with the highest magnitude, representing me actual 
edge. 3;a shaded. 
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Sobei edge magnitude values 
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Figure 7£ Edge magrtiruce values computed with me Sonet operators 



Angle image 



The Wge detection functions produce an imago containing tno angle of each edge DtxoL 
If xis the honzcntal edge component value and yis the vertical edge component value, the 
edge angle ia the ccuntwciockwit* angle kwtu^wn (ho horizontal axis and tne magnitude 
vector. " 

Figure 77 snows me geometric relationsnio oetween the edge angle, tne acge magnitude, 
and the two edge components. !n this figure, M is Tie magnitude vector ana 9 is the edge 
.angle. 



VertloaJ ad go 
component 




Horizontal edgs 
component 



Figure 77. Geometric interpretation of eage angte 

Edge angies are stored In the ancle mage in Dinar/ form. They are represented as 
fixect-ooint nurroers with tne cecimai point to the left of tne most significant bit and are 
interpreted as tractions of 360°. Ycu can choose tne maximum numoer of die avatiaoie to 
express an angte; tnis value must be less than or equal to a. If n is the maximum number 
of 'bits used to express angle, and x is the binary representation of an angie..ycu can 
-convert x to degree notation oy using tne foitcwihg formula; 



angie(x) = 



360x 
2* 



Note that with a S-bit representation, angles can be expressed to tne nearest 5.625°: with 
a 7-blt representation, an angle can be expressed to the nearest 2.3°: with an a-bit 
representation, an angte can be expressed to the nearest 1.4°. 



SUBSTITUTE SHEET (RULE 26) 



W ° 98/18117 , FCT/DS97a826S 



4 Edge Detection Tcol 



The edge angle is computed using me arc angent function if is the horizontal edge 
component and y is me verocal edge component for a pixel, (he eage angle for that pixel 
ia calculated as shewn in Tabla 3. 







x>o | y > o 


arctan( y/x ) 


x > 0 


y < 0 


360-arctan( ly/xl ) 


x>o y = o 


0 


x < 0 


y > 0 


i80-arciari( iy/xi ) 


x < 0 


y < 0 


1 80 * ar ctan< y/x ) 


X < 0 


y=*0 


180 


x = 0 


y>0 


90 


x = 0 


y < 0 270 


x = 0 


y sO | 0 



TabtQ 3. Formulae tor ccmouting acge angles 
The computation is designed so mat an ecge angle e is in me roncwmg range: 
0*^e<360° 
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Figure 73 contains six rows, each row contains (from left to ngnt) an imago with the central 
plxei shaded, the venical and horizontal ecge component value of tne shaded DJxeJ, the 
image w.tn a vector auoor imoowad crowing rn* ooga angla ot ma central dukaI. and th« 
tcrmuia usea to compute tne ecge angie. Nonce that the ecuauens defining tne edge angle 
vaiuA are designed so :hat me vGctor always coma to the oricnter side of tne edge. 
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Figure 73. Camming me sng/e ot various aagss 
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By seeing a flag tn tne acge detection parameters, ycu ;an soecsfy that you want angles 
computed in the range 0° to 180° msteaa cf tne cemult range, 0° to 360°. When you select 
mo smailer range, an edge angle nthat is greater man 1 30° macs :o aoga angle (n— 180°). 
For insanca. 3CC* rr.acs to 120°, and 2?C° macs to 9G°. 

Use ma reduced angle range when you want r.e same a eg 8 angle results regardless of 
me polarity of me image. As shown in Figure 79. wnen me range or angles ts restricted to 
me range 0° to 180°. me same ecge angies ara ccmcutec for a oiaric object on a wnite 
background as tor a white object on a biacK saexcrcund. Whan ms range of angles is 0* 
to 360* (also Figure 79) this is not the case: reversing tha pcianty ot tne image reverses the 
acge pixel angle. 




Ftgura 79. Eftac: ot reversed cctarity cn 9age angles tor tne two ranges 

The binary representation ct an angle in tne range 0" to 1 8C° is similar to me representation 
of an angle in the range 0° to 260°. if n is me maximum numoer of bits used to express 
angle and x\% the binary representation cf an angie in me range 0° to 180°. you can convert 
xto cogroo notation by acoiying tha following fcrmuJa: 

130* 
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Edge Detection Preprocessing 

If you succiy a oixei too. tne edge detection lure sen -jew it to crerroceaa your incut 
image before It calculates me hcrteentai and vertical ecca comocnents. rcr a cescnctlon 
of oixai maooing. 9M enactor 3. A'*©/ fUscoing. 

Edge Detection Postprocessing 

■" Although acge angles are defined fcr ait pixels in an image, tney are signrticant cniy at tne 

pixels with me greatest scge magnitudes. There are many ecges oetween pixels wnere tne 
grey levels on either sice of :no ecge differ by oniy a few percent. These insignificant edge 
pixels, due to frame gracoer ncise ana ranccm fluctuations in oixel vaiues. can clutter ycur 
ecge megnituca image end make it difficult to evaiuato tne data in mat imago. Aieo ©oge 
magnitude values usually increase gradually in a "smear* as tney aoprcacn an ecge ana 
than docrwea ac th* acga >c crccz&d. You might want to cnaroan mo magnituco imaga by 
fiirering cut some ecge pixels that are near, out net at. an ecge. 

;.t. _ . . The Edge Oetecccn tcci supplies three postprocessing metnods. Two of tnese metnods. 

- — - * - setuo and run-time tnresnoicing. set tne lowest edge magnitude oixeis :o 0. The tnira 

- - ■ method, peak detection, takes an acge magnitude image ana processes it so mat. for any 
small Dixel neighDornocd. only the highest adge magnitude pixeis remain and tne other 
pixets are set to 0. See tne section Pea* Derecvcn on an Ecge Magnitude image cn 
page 229 for a ccmciete discussion of mis metnod. 

After postprocessing, ecge angles corresponding to :sro edge magnitude pixeis are also 
" set to Q uniess you have specified run-ame tnresnoicing cut not peak detection. In this 

case, edge angies corresponding to zero edge magnitude pixels can be nonzero, du: 
should oe considered undefined. 

S«tup Thro* holding 

You can supply a setuo time rejection oercantage. This is a vaiue n that represents a 
percentage of 'tne highest oossicie edge magnitude value. With this metnoa.jany 
magnitude values in we first n percant of ail possible vaiues are forced to Q. For example, 
rf you provide 6 bits to express magnituce values ana specify that the icwest 5 percant of 
. ..magnate values are torcad to 0, then any magmtuca Delcw 3 is forcaa to 0 stnea 5 
percant cf 63 (truncated) is 3. 

Run-Ttmo Thr»shoWIng 

Ycu can suppiy 3 run-ttme reiecden percentage. This is a vaiu* n thai represents a 
percentage cf tne r.ignest actual ecge magnitude vaiue in a magnitude image. With this 
method magnitude vaiues in the first n oercant of the magnituce vaiues in a specie image 
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are faced ic 0. for example, if the nighest magnitude value in a particular image is 51 . and 
if ycu soeciry mat me lew est 5 percent of tne run-time magnitude values are forced to 0, 
then any macnitude beicw 2 is forced so 0. sinca 5 oercant of 51 (truncated) is Z 

Using a setup rejection percentage is raster Ehan using 3 run-time rejection percentage, 
sinca the values ;nat are tcrcsd :o 0 are computed onca. Defcre any magnituce images are 
created. Hcwever, me run- lime metned is more flexible and rcpust. If you are producing 
magntuce images or me same scene cur under varying lighting conditions, ycu snouc use 
a run-time rejection percentage since me proportion of magnitude values forced to 0 
remains staoie as me range of magnitude values changes (due to varying illumination). 

Po*tproc»3tn3 with P*ak Detection on a Magnrtuo* Jnag« 

Peak oetecricn ia a merhoa of filtering an tmege to that only peek pixels remain and other 
pixels are set to 0. Peak cixeis nave a vaiua greater man or equal :c some or ail or tneir 
neighboring pixeie' vaiuee. 

Ycu can set a flag in me ecge detection parameters structure mat instructs the edge 
detection function to run peak detection on the magnitude image during Dostcrccessjng. 
Ail ncnDeak pixeis are filtered cut of the magnitude image and the edge angie image before 
they are returned, When ycu specify mis. ycu must supDiy pea* detection parameters in 
the edge detection parameter structure. 

The ecge detection function does oeak cetecnen postprocessing after it has ccne setup 
ana run-time threenciarng. There wiil be a 2-sixei-wlde corcer of zoroQ in a paax-aoiootod 
magnitude image. 

The section Pea* Dawctton on an Ecge Magnituce image on page 229 discusses, in detail, 
the benefits of using peak deteocn on an eoge magnituce image. This secticn also 
describes me crcper way to parameterize Deak detecticn so mat it niters ecca magnitude 
pixels correctly. However, you co not need a detailed knowledge of peak detection to use 
it with acge detection. Suggested oeak detecticn parameters are succiied in tne section 
Data Structors cip_ecga jzarams an page 199. 

Figure 80 contains an image, its acge magnitude image, and the image mat results Ircm 
peak detection. The edge msgmtuca image contains a smeared edge, which is sharpened 
■ ' by.peak detection. 
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Figure 30. Peak detecting an edge magnitude image to determine aczuat 
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Sample Application: Angle Histogram 

You can uo* mo magnituaa and angi© image© of an oojoot to croat© an sngio histogram of 
mat object. This rs a table that assigns a value n to each ecge angle 6 if tnere are nedge 
pixalc (aoovo a givon magnitude throshcid) whoaa edge angle is a. 

The angle histogram supplies a useful "signature* of an object's shape. You can use mis 
signature to gauge similarity among objects, to detect rotations of an object, and to detect 
Daws in an object. 

Once you have created eage magnitude ana edge angie images for an object, you can 
create and aiapiay an angle histogram ae foilowa: 



void ar*gla_iistcgr3-*n ( res - threshold) 

cip_edge result Tes; /• results structure pointer * / 

int threshold; significant edge threehold •/ 

{ 

int i,j,:nag; 
int hist [256] ; 

cu^clear (hi3t, sizeof (hist ) ) ; 

tar (1=0; i<ras->taag.Ung_p->wldtr.; i**) 

for ( j=0 ; j<r9s->mag_isig_?->heightt; j* + U 

aiag=rea->niAg_lmg_p->g«t (r»3-»roag_ia:g_p, j ) ; 
if (mag > threshold) 

hist[ras->ang_img_p->9et (re9->ang_i»g_ p,i. j) 

) 

cgr.hi atograun ( c Aq_itia9 ft • c * 55 i : 

caq_ display ( ) ; 

return; 

) 
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Figure 31 contains four snaoea; aach shape Is paired with its angle histogram. Note that 
tne square and the plus sign have identical angle histograms. The angle histogram of tne 
c:rc:e is Jlat oecause mere is no dominant angle along tna ecge of a arc!o. 




Angle 




0 90 180 270 260 
Angle 



Figure 37. Assorrea snaoes wim rneir angle Histograms 
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Figure 82 demonstrates tne affect of rotation on itie angle histograms of a plus sign. The 
histogram shifts as the plus sign rotates. 
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htgure 82. Effect of rotation on an&e histogram 



42 



SUBSTITUTE SHEET (RULE 26) 





WO 98/18117 



PCT/US97/182S8 



Caliper Tool 4 



The Caliper Tool 



This seCwCn carinas rns Caifcer Tcci anc essences :n general terms hew it wcrks. The 
ccncacts introcucac in :his asc::cn are cesensac in detsil in later Mctxrc 



The Caliper loci is a tec* for locating edges and edge pairs in an image. The edge of an 
object in an :mage is a cnarge in gey valus from carK to lignt cr !ignt to dark. This change 
may 8Dan cavoral d»xgIc. The Callc-ar Tod provioa* .Tiorhodo for ignoring dog a a that are 
caused by noise cr tnat ar9 net cf interest for 3 cemcular application. 

The Caliper Tool Is modeled on :he mechanical engineer's cafioer. a orec:sion device for 
measuring distances. You soecify tne seoaraEicn between the call c or 'jaws.* and tne 
Caiioer Tool searcnes an area you specify for sdge cairs separatee oy that distance. You 
can aiso searcr. for incivicuai ecges wnen you Know meir approximate location in an 
Image. This is similar to using a caiioer to measure deptn. 

A Typical uw tor the Caliper Tool ie pan inspection on an aeeembly lino. For example, in 
integrated circuit lead inspection, a cart is moved in Ircnt cf a camera to an approximately 
knowvn location (the axo&c:aa lacancn). Ycu use tno Calicor Tocf to aotormme the exact 
location cf tne ieft acge cf :ne pan by searching for a single lignt to dark aege ai the 
exoectea iccancn (see Figure 1 03). The eccs that is cicsest to the expected location is the 
left ecge of -ne pan. 



The Purpose of the Caliper Tool 



Saarcn 



Image 



area 



1 




Figura 103. Tha Caliper Tool is used to locate the left $6g8 cf the part. 
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Once ycu Sr.c :r.a :en acgs cr u-s sari, ycu car, srecic: wnere ins leads should lie within 
tne tmaga to 3csc:ry a wince* in whicn :o iccx !cr [sacs (see Figure 104). Ycu can then use 
the Caliper Toci *o searcr. fcr ecge catrs ccnsisar.c of a aa/K to licht transition followed bv 
a Iignt :o da/K transition, anc separatea =y Tie exceced ieaa wictn. With information 
caicuJatea oy tne Cancer 7oa, ycu zzr. ccmoara measures iead wicth and location to 
exoectea isac wicin anc iceanen iz make an accaot/re;ec: aec;sion acout the pan. 



The Cancer Tec: uses zrqac:;cn 10 xac a Nvocimenswnai wincow of an image (the caticer 
window) into a cr.9-aim8nsjonai mage. Protection ccilacsas an image by summing the 
pixeis in tne cirec::cn cr :r.a srctecticn, wntcn :er.zz :c amciiry acges :n mat Olfaction. 
Figure 10£ shows an image, a caiioer winccw. the one-directional image that results from 
projection, ana a grasn sf tr.e oixei grey values :n Tie cne-omensional image. 



Sesrc.n 



Image 




Figure *C£. The Calmer Toci car? oe used to measure iead width and 
'.sec zczc:ng an an snrsgrstsc circuit. 



How the Caliper Toci Works 
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Image Calioer window 

t I 




First ecce Second ecca 



Figure ICS. Overview of the Calioer Tool 

Ycu may scecify rmt the Caiiper Tool oerform pixel mapping bntcro projection. PixeH 
mapping can be used to filter an image, attenuate or amplify a range of grey values, ana 
otnoiwiee mac Dixei value©. Sao Chop tar 3, Pixel Mapping, in me Image Procacs/ng 
manual for a complete cescnption of pixel mapping. 

After projection, an edga fitter is aopiied to me one-dimensional image to further ennance 
edge mfcrmaticn and to smcctn rbe image by eiiminating miner grey value changes 
oerween neighboring pixels mat are most likely caused Dy noise. The edge filter produces 
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a first derivative image (the most rapid cnanges in grey value in me prelected image result 
in the hignest peaks in me fiiterec imag9). Figure 106 shows me cne-cimensicnai image 
from Ficure 105. an imace c&ieratsd oy me ec^e flits/, ana a gracn cf mat image. 



J I 




First edge Secona edge 



Figure we. An eage fitter :s acptiea to me one-amensicnzi image that 
results trom projection 

After me filter operation, ecge detection is performed on the filtered image. Eage cetecticn 
is a method cf evaluating peaks :n the filtered image and ignoring eagee tnat are not of 
interest, sucn as sages representing nose in the image. Edge cetecacn applies geometric 
constraints tnat you specify to eacn ecce or edgo pair found m the image. Acotying 
geometric constraints provides a way to limit the number of edges ro evaluate oy assigning 
a score to each ecge or ecge pair oased on such factors as the axosctea location of me 
edge or ecge pair, the distance oetween edges in an edge pair, and tne minimum grey 
levei difference oetween pixels on each side of the edge. 

The Caliper Too*, returns as many results as you have specified in the ccip_p«rams data 
structure anc computes me geometric average of tne score of eacn geometric constraint : * 
that ycu specify. For eoges or edge pairs whose score equals or exceeds the accept 
threshold. ihe Caliper Too* returns information sucn as me score ana location of me cage 
or sdge pair and me measured distance oetween edges in an edge pair. 

The controlling variables cf an edge or edge pair search define a caliper, and include such 
information as me cimensicns of the search window and me edge detection scoring 
parameters. The type of r.a data structure mat defines a caliper is ec!p_cai)per. 
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The Caliper Window and Projection 

This section describes how you specify me caliper wndcw. which is tne subset cf an image 
in which to locate ecges cr edge pairs. It men describes aroj&cscn, which is sna mapping 
of me cdiper windcw into a one-dimensional image. Finally it describes the adga filter. 
which enhances edges in the one-dimemicnal imaoe. 

Caliper Window 

Th* cAiicerwineGwia the aoraon cf an image in whlcn *ho Caliper Toci csorcnoc for edges. 
It is cefined by the search lengtn and orcjecrton Jengtn, wmcn are tne wicm and height 
resoectiveiy. of the window in an image that will be projected into a cnG-cimensonfli image. 
The caliper window is Hiustratea In Figure 107. 



Figure ',07. The Cahoer Tool window 

The Dixei in the center ot tne caliper wincow ts callea the applicsticn point. This is the point 
in the run-time image at which ycu want to locate edges. 

Caliper Window Rotation 

For many applications edge information is needed at more than one angle in an image. For 
example, to inspect a rectangular pan, ycu may want to measure the dimensions of the part 
.by projecting the imace first at 0° to measure length, and then at 90° to measure wjdth. The 
"caliper window can be oriented at any angle to locate edges in an image. Angles increase 
In the clockwise direction from tne horizontal axis. 

Figure 1 C8 snows now changing the caliper window angle affects the projection and edge 
soarcn directions at 90* rotations. The Caliper Toot runs raster for 90* rotations man It does 
for arcitrary rotations. 
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Ecae oetection 
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F?gure 7 05, Tne cal/per wmacw at 90° rotavons 



Skewed Window Projection 



Figure 109 snews a -IS 3 caiicer window with window rotation disaoied. !n tn:a case, the 
Caliper Tcot creates a "skewed* winccw anc z rejects it atcng the angie at skew into a cne- 
aimenacnai image. 
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Image 

* 



Searcn length 




Figure ICQ. Skewed window projection 

When winaow rotation is disacied. you specify how tne skewed window wiil be orciecrad: 
wim or witncut interpolation. interpolation increases accuracy at :ne expense of execution 
speed. 

When you use a skewed window without intercalation fcr projection, me pixels m the source 
image are summed along the angle oi anew as shown m figure 110. Each otxet in the 
two-dim enscnai image ccnmoutes to one Qixel in the one-dimensicnai image. In 
Figure 110, !how otxeic containing rns numcor 1 ccntnouta to tne deeunancn pixel 
containing me numoer 1 . ana sc cn. 
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Figure 110. Skewed window prcjecjion without interpolation 

When yaj use a skewed winccw wttn inters olaticn for ofcjecticn. tne skewed window is 
transformed into a recanguiar winacw oercre projection. The uansicrmauon is performed 
by clp_tr ancf orm(), wnicn uses several neighocnng pixels from tne two-dimensional 
image ro calculate me cixei value for the cne-dmensionai image. See Chapter 2, Basic 
image Processing, in me /mage Processing manual fcr a complete description of the 

function otp_tran«torrn(). 
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Rotated Window Projection 

Figuro 1 1 1 chows a -16° oaiip* window wim window rotation orublod. In mi© ca&o tno 
Caliper Tool rotates Tie >wo-dimens;cnai image anc projects rt into a one-dimensionaJ 
image. 




Figure in. Rotated window projection 



Choosing the Projection Type 

The c~cic- of $*eweao< rstarecorciecacn aeoencs cn me angie cf the edge cr edge oair 
cf interest Tn me image anc :ne remaining content of the image. If me rotation angle is close 
to 90° or if a skewed image wiil contain encugn of the eoga or edge pair of interest to 
generate strong edges, ycu can use skewed projection to speed program execution. 
Figure 1 12 snows an image, a caliper window and the one-directional image mat results 
from projection. The skewed winccw contains enough edge information, so skewed 
projection may be used. 
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Image 




Calicer 



wincow 



One-dim ensicnai 
intGnsity image 



figure ? 12. Skewed projection may be used 

Figure 1 12a sncws a caliper winacw wnere tne ecges of interest are at 15° ana winccw 
rotation is disabled. A wincow targe enough to enclose tne9e ecge9 wcuic also include 
mucn of tno dark area in tne image, wnicn in some cases wcuia result in a ono-aimonsional 
image wnere tne ecges are ooscured. 

Figure 11 2d sncws a calicer winccw at -15- in tne same image witn winccw relation 
enabled. Only tne ecges of mTeresi are included in mis window. 
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Figure 1 13. Skewed projection (a) ana rotated protection (P) 



Caliper Tool Optimization 



When setting up me vanaDles for a caliper, you specify the following types of optimization: 

• Space optimization: The Caliper Tool is optimized for minimum memory use at the 
expense of execution speed. 

V Spaed optimization: The Caliper Tool will use's number of internal techniques to 
improve execution speed at the expense of increased memory use. 

• Oefadt optimization: The Caliper Tool will weigh soace and soeed equally, using 
citghtly mor© cpac© than if you cpocify cpac* optimization and elightiy ciow«f 
execution soeed tnan if you specify speed optimization. 
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The Edge Filter 



After projection, an serge tiiter ts run on the resulting one-dimensional image. The edge fitter 
accentuates edges in the image and produces a filtered tmaga. The peaks in mis image 
indicate strong edges. 

The two parameters of tne edge fitter are its size and laritancy. Edge filter size is the number 
of pixels to each side of the edge to.eonsider in the evaluation. Edge filter leniency is the 
number of pixels at an edge to ignore between light and dark sices of the edge (leruency 
is described in tunner detail below). Figure 1 U illustrates an edge filter. The edge filter js 
ooainoned over the leftmost pixel m the one-aimensonal imoge where it will entirely fit in 
that image. Pixel values to the left of the leniency region are 9ummea and subtracted from 
iho cum of the ptxet valuee to the right of the leniency region: in mis exempie, (O-rS)-(O-O) 
yields 5 for the edge filter image. 
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Ftgure 7 14. Tho &ago fitter jb positioned in the ono-dimenatonai image. 

As shown in Figure 115, the edge filter is tnen applied ai each successive pixel in me 
one-dimensional image until the edge filter no longer fits entirely in the image. At each pixel 
position; me pixel values to the left of me leniency region are summed and subtracted from 
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the sum of the pixel values to me nght of the leniency region and the result is written to the 
edge filter (mage. The resulting image Will be 2 • si2e + leniency - 1 smaller than the 
one-dimensional image. 
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Figure 115. The edge filter is applied at each successive pixel until the 
edge niter no longer fits entirely tn tne image. 

Figure 116 shows a graph of the one-dimensional image and the edge filter image from 
Figure 11 5. Pea* position is calculated to suoptxei accuracy. 
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1-D image 



Edge filter image 



Figure 1 16. Edge fitter imaga graph vs. one-dimensional Image graph 

The graph in Figure 1 1 6 represents an eoge niter apolled to an iceal image; few 
aopiications wouJd result in as ideal a graph. All Images contain some degree of noise, and 
a typical edge in an imago can do several pixria wide. The next two sections provtce 
guidelines tor optimizing the edge filter by choosing appropriate values for size and 
leniency. 

Choosing Edge FiltBr Size 

Eoge filter size specifies me numDer of ptxeis on eitner side of an edge To consider in an 
edge evaluation. Size should usually oe greater man l Decause noise in an image causes 
grey level differences between neignooring pixels. Figure 117 snows a grapn of a 
one-dimensional image of a light band on a dark background, and a graph of an edge filter 
image where a si2e of 1 was used. Because there is noise in the image, many more peaks 
exist than are of interest. 



56 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/US97/18268 



CallperTool 4 




Edge filter peaks 



Figure V7. An sage filter size of i is typical fy too small. 

increasing size tends ro smootn me ecge niter image oecause summing several pixels on 
either side of an edge rends to reduce the effects of noise. Figure 11 8 shows an image 
similar to The one in Figure 117 where a size of four was used. The only two peaks in me 
image are the peaks of interest. Size is typically between 2 and 5 inclusive. 
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Edge filter peaks 



Figure 7 7& Using four for edge filter size reduces the effects of noise on 
ihQ edge filter. 

Choosing Edge Filter Leniency 

Edge filter leniency specifies a region of pixels to ignore at an eage in an image. Due to 
slight changes in edge or edge pair position or orientation, a pixel on the edge of a feature 
may not always have the same grey value. By setting a leniency zone between the light and 
dark areas of a feature, the effects of slight changes in feature location on filter resiits are 
minimized. 

Figure 119 shows a graph of a one-dimensional image with two edges that span two pixels 
eacn. and graphs of the output of two edge filters. The first edge filter has a leniency of 0 
the second edge filter has a leniency of 2. The edge filter with a leniency of 2 produces ' 
higher peaks. Two is a typical value for leniency. 
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Edge 




Figure 1 19. The effect of leniency on an edge filter image 
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Registration Tool 



This chapter describes me Cognex Point Registration tool. It contains seven sections. 

The first two sections, this one and Seme Useful Definitions, provide an overview of the 
chapter and define some terms that you wlf) encounter as you read. 

Point Registration Tod Ove/v/ew provides information about tne capabilities and intended 
use of tne Point Registration tool. 

Haw rfia Point Registration Toot Works provides a general description of the operation of 
the Point Registration tool. 

Comparing the Point Registration Toot with Search compares the Point Registration tool 
with the Cognex Search tool. 

Using the Point Registration Toot describes some of the techniques that ycu will use to 
implement an application using the Point Registration tool. 

Point Registration Tool Data Structures and Functions provides a detailed descripnon of the 
data structures ana luncnons that you will use to implement your application. 
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Point Registration Tool Overview 

The Cognex Point Registration tcol per terms fast, accurate pcin: registration using two 
images mat you supply. 

Point Registration 

Point registration is the prccsss cf determining me crecise orfsat between two images cf 
the same scene. 

You use the Point Registration icci ro pericrm point registration by providing a model image 
and a target image, aiong with a starting point within the target image. The model image 
snouid oe a reference image of me feature you are attempting to align. The target image 
should be larger than me mccel image, ana it shouid contain an instance of the model 
image wtmin it 

The Point Registration tool will cecermine the precise location of % ha mccei image within the 
target image. The Point Registration tccl will return the location of mis march, called the 
registration point with suooixai accuracy. 

Figure 160 illustrates an example of how the Point Registration tcol performs point 
registration. 





unHoi Tamm ^rxn A Location of model image 

Model image Target image wimin tarnAf . ■ 



within target image 

Figure 160. Point rggtst raven 
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Some Useful Definitions 

point registration A search technique designed to determine the exact pant at 

which two images of the same scene are precisely aliened. 

global minimum The lowest value in a function or signal. 

local minima A low value in a function or signal. 




Local minima Global minimum 



subpixei accuracy Positions within an image may be specified in terms of whole 
pixel positions, in which case the pocirion refers to the upper- 
left comer of the pixe*. or in terms of fractional pixel positions, in 
which case the position may He anywhere within a pixel. 
Positions specified in terms of fractional pixel positions are 
referred to as having subpixei accuracy. 
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The Point Registration tool is optimized to locate precisely me model image wimin the target 
image, even rf me target image contains image defects such as reflections, or if the modei 
image is partially obscured by otner features in me target image. Figure 161 illustrates an 
example of point registration wnere r^jarget image is partially obscured 




Figure 161. Point registration with partially obscured target image 
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How the Point Registration Tool Works 

The Point Registration tool finds the location of the mode) image within me targe! image. 
You ooerate the Point Registration too! by suDOlytng me model and argot images, atong 
with the iocation within the target image where you expect the origin of the model image to 
be. The Point Registration tool will determine, with subDtxel accuracy, where the origin of 
me modei image lies witnin me target Image. 

The Point Registration tool works by computing a score indicating the degree of similarity 
between me model image and a particular portion ot the target Image that is me same size 
aa me meed imago. This score can range from o to 253, witn a score of 0 Indicating that 
me moaei image and the target image are perfectly similar and a score of 255 incicating 
that tho model image and me target imago are perfectly dissimilar. The tool computes mis 
score for locations in the immediate neignborhcod surrounding the starting point. The tool 
will find the location within tnio notgnborhood of me target image that produces tne fecat 
minima in the vaiue of tnis score. 

By adding an interpolation step, me Point Registration tool then determines the iocation of 
me mode* image within me target image with subpixel accuracy. For typical images, me 
Point Registration tool can achieve accurate registration to within 0.25 pixels. 

Bocauc* of tho way the Point Registration tool seeks the local minima, rf the starting point 
you specify :s more man a few pixels rrcm the actual registration point, tne tool may not 
return tha corroct registration ooint. Tho exact amount of varianoe that the Point 
Registration tool can tolerate will vary cepending on the images. The variance may be as 
small as 3 :o 5 pixets for some images or as large as 30 pixels for others. 

Point Registration Score 

Eacn time it ia invokea. the Point Registration tool will return, in aaatlcn to tne location of 
the origin of the rnccei image within the target image, a score indicating me degree of 
similarity b«rwo©n the modol imag© and the target image. The ecore returned by the tool 
will be from 0 to 255, with a score of 0 indicating that the model image and the target image 
are perfectly similar and a score of 255 indicating tnat the model image and the target 
image are perfectly dissimilar. 

If your point registration receives a high score, the actual precision of the point registration 
location may be somewhat lower than if the point registration receives a low score. 

Exhaustive Point Registration 

When you use me Point Registration tool, you supply the tool with two images and a starring 
location within the model image. The tool will confine its point registration search to a small 
area around the starting location that you specify. 
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The Point Registration tool can also oerform point registration exhaustively, that is. by 
computing the score for every possible location cf the mccei image within the target image. 
This procedure, called exhaustive cant registration, is extremely slew. It can be helpful, 
however, in debugging applications where the Point Registration tec! dees act acoear to 
be working correctly. 

Masked Point Registration 

You can limit the areas of the mooal image ana target imaga mat the Point Registration tool 
uses to perform the point registration oy supplying a series oi rectangles to tne tool. If you 
supply these rectangles, the tcci will compute the score oasac oniy on tr.cse oixeis 
contained within the rectangles mat you scecify. 

Figure 162 illustrates an example of specifying a masking rectangle, in ins example, tne 
right side of the model is often ocscured in the target image. 3y specifying a rectangle that 
covers the left eide of the modei Image, you can cause the Point Registration tod to 
consider oniy the pixels in thai part of tne model image. This wilt tenc tc increase the 
accuracy of tne point registration. 



Masking 
rectangle 





Modei image 



Target image 



Location of model image 
within rarget image 



Figure 162. Masked point registration 

You.can a)90 specify several masking rectangles. Figure 163 illustrates an example where 
tho o*ntar of the mobeJ image te often obscured in the target image. By specifying a series 
of rectangles, you can eliminate from consideration tne part of the image that is most 
freausntly obscured. 
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Masking m 
rectangles 





Location of model image 
within target imaoe 



Model image 



Target Image 



Figure 163. Mssxoa point registration uatng muivpie rocrartgt*9 



If you specify muincle masking rectangles and the rectanoles ovenao. any oixets that are 
contained in more than one masking rectangle will be counted toward the score multiple 
times, once for eacn rectangle in which they are contained. 

In all cases, the Point Registration tool wiil return me locanon of the origin of the model 
image witnm the target image. 



The Point Registration tool may not work well in cases where the model image and the 
target image have different intensity values. You can perform limited image processing as 
part of tne point registration by supplying a pixel mapping table. 

If you supoly a pixel mapping table, avery pixel in the target :mage will be mapped to the 
value in me pixel mapping table at tne tocaoon within the pixel mapping table given by the 
pixel's value. For example, tf a Dlxel in the tar gar image had a value of 200. it would be 
mapped to tne value of tne 200* element within tne pixel mapping table. 

IT you nave specified a masking rectangle, only those portions of the target image that lie 
within the masking rectangle or rectangles will be mapped through the pixel mapping table. 

For mora information on pixel mapping, see Chapter 3, P;>e/ Mapping, in the Imags 
Processing 'manual. 



Image Normallxatlon 



66 



SUBSTITUTE SHEET (RULE 26) 



# 



WO 98/18117 PCT/US97/18268 



6 Point Registration Tool - 

Point Registration Tool Data Structures and 
Functions 

This section contains descriptions of the Point Registration tod data structures and 
functions. Ail Point Registration too! data structure and function names begin with the orefix 
creg_ 

Structure creg_params 

creg_jMnmw contains parameters that determine how the point registration searcn will be 
conducted. A structure of this type is suppllea as an argument to creg_pokit.r»gtster(). 

# include <register.h> 

cypadaf a cruet 
( 

c_Int32 start.x. 

«fc»rt_y/ 
c_Int32 recc_count; 
char *option*l_jmp( 
c_Int32 flags; 
) crea_parajns ; 

• staruc and $tart_y are the x- and y-coordinates of the starting position for trie point 
registration searcn. 

• rect^counr is the number of rectangles in me rects argument to cn»a_polnL_ragl«tor 0 

• opuonaimap is an optional pixel map. If opuonaljnap is non-NULL, the Point 
Registration tool will replace each pixel in the target image with the value contained in 
the element of opuonaljnap corresponding to the value of the pixe< in the target 
imago. If you supply a value for optionaLmap. you muat supply an image of type 
FAST6. 

You can use opttonai.map to compensate for image-tp-image brightness variations. 

nags specifies me type of point registration mat me Point Registration tool whi perform. 
flags is constructed by ORing together any of the following values: 

• CREG_TYPE 1 ia reserved for future use; you should always include CREGJPfPE 1 
in me value of flags. 

• CREGJYPE2 is reserved for future use; you should never include CREGJTYPE2 
In the value of flags. 

• CREG.NORMAL is used to specify a point registration based on finding me local 
minima of the score value. 
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• CREG_EXHAU$TIVE is used to specify a point registration based on finding the 
global minimum ot me score value. 

if you specify CREG^NORMAL and if stan_x and starLy are more man a few pixels 
from the actual registration point, the tod may return a location that does not represent 
me Dest maicn wltnin tnc Image, if you specify CREG_EXHAUST1VE, the registration 
will return the best registration match for the entire image. An exhaustive point 
registration will be extremely stow. 



cf5g_po4nij-*gi«w returns a pointer to a crag_re*urU structure. creg_po|nt_r«gl«ter 

fills in the structure with information about the results of a point registration operation. 

•include <register.h> 

typedef struct 
( 

float 

y; 

c_Inc32 ecore, 

time; 
char on_edge_x, 

on_edge_y ; 
} creg_re suits , 

• x and y are the x-ccordinate and y-coordinate. respectively, of the location within the 
model image at wnicn the target image was found, x and y give the position with 
subpixol accuracy. 

• score is a measure of the similarity between the oixel values in me model image and 
tne target image at me nearest whole pixel alignment position, score will be between 
0 ano 255. with lower values indicating a greater degree of similarity between the 
model image ana tne target image. 

• time is tne total amount of time that the tool spent on this point registration, in 
milliseconds. 

• " on_edgB_x is sat to a noruero value if, along the x-axic, ono odgo of the model image 

area Is at the edge of the target image. If on^edgeje is nonzero, the accuracy of the 
position information may be slightly reduced from what it would otherwise be. 

• or^edge^y is set to a nonzero value if, along the y-axis, one edge of the model image 
area is at the edge of the target Image. If on^aage^y Is nonzero, tne accuracy of the 
position information may be sJightfy reduced from what it would otherwise be. 



Structure creg. results 
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Using the Point Registration Too! 

This section describes hew to use (he Point Registration tocl. 

Obtaining Model and Target images 

You acquire the images that you use as model and target images using the techniques 
described in Chapter 7. 3000/4000 Acquisition and Display, in the 3C0O/4CCO Image 
Acquisition and I/O manual and Chapter 1, 5000 Acquisition and Display, In the 5000 
Image Acquisition and I/O manual. 

You should acquire all images mat you plan to use with the Point Registration tool, 
especially images of scenes with low contrast, using a bit depth of at ieast 3. For fastest 
performance, you shouid acquire images as FAST8 images. 

When you acquire the model image, you should take care to ensure that the image is as 
idea) a model as possible of the images that you will be using as target images. The Point 
Registration tool is designee to align target images with a wtae variety of image defects, 
but in order tor the iocJ to work, tho model image needa to be bo free from defects ae 
possible. 



Specifying a Starting Offset 

When using the Point Registration too), you must specify a starting location in the mcdei 
image. The tool will perform its point registration search starting at the point that you 
specify. Because the tool only seeks the local minima for the score value, if you specify a 
starting location tnat van go greeUy from the actual registration point, the point registration 
operauon will fail. 

You can use other vision tools such as the Search tool or tne Insoection tool to locate me 
modei feature within the target image, or you can rely on operator input to specify the 
coarse location of the feature. 
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Comparing the Point Registration Tool with Search 

This section compares trie Point Registration tool with tna Search tool. For more detailed 
information on the Searcn tool see Chapter 7. Searching, 

The Point Registration tool has me following advantages over the Search tool: 

• It rs less sensitive to images that contain areas of pixels that have widely different 
values than the same pixels in the model image (i.e.. blotches or occlusions). 

• ^requires no training step. 

The Point Registration tool has the following disadvantages when compared with the 
Search tool: 

• It is capable of tine alignment only. 

• It does not work with scaled or rotated target images. 

• It does not work well with images with brigntness changes. 

• - It will not find or rank multiole registration oants. 
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Function creg_point_registerO 

crag point raqlatarQ perfenne point ragtatraoon ucir.g tr.a mocd imago and target image 
that you supply. The point registration will be controlled by me parameters :n tne 
efe g p a/am* structure succiied to me tuncacn. cre^_po\ntJ9^i*t»f() returns a ocintar to 
a cr»g_resuits structure that describes the result of the point regstraficn. 

t iaeluda <regiater.h> 

crsg_r asulcs •crag_?oinc_rsgi9ter(conrc cip_buiiar *rargec, 
const: cip_buffer • model , const cia_recr *rects . 
consc crag_paraias 'paranja, creg.resuicj Tasuits) ; 

• target octnts to ma image to usa as tne ttrgat rmago for !hic point rogictraoon. target 
must be at leas; one pixel larger :n bctn tne x-dimanscn anc me y-cimensicn tnan 
model. 

• model pcinrs ro the image to use as the mocal image (or tms cc:r.t registration, model 
must be at least one pixel smaller in both the x-cimens:cn ana me y-d:mension than 
target 

• rec& points to an array of cia.raci structur es. It rects is non-NULL the Point 
Registration tool will limit me comparison of pixel values :c just these cixeis that lie 
wimin me rectangles comainea in rects. lime rectangles in recrs ovenac, a otxei will 
be included in the ccmpanson onca for eacn rectangle mat certains it. 

• params points to a creg_parmms structure tnat defines me parameters to use fcr this 
point registration. 

• results points to a creg_re*urt structure. crag_pcint_regi«ter() wiil claca the results 
c* this zcint registration m rasuJts snc wiil return a pointer to /acuite. 

if you supply NULL tor resuiis, crog_point_ragi«t»rt) will allocate a crag_reeult 
structure ircm the neao using the default ailccatcr. Ycu can free mis structure by 
catling fr©e(). 

erBg_polnt_regJ«erO mrews CGEN_ERFL5ADARG it 

a rects is NULL and me recLcount field of params is net 0 

• If model »9 larger man or the aamo aiza as target 

• If me start je field In params is less than 0 or greater man the wictft of target minus me 
width cf model 

• If tne sian_y field In oaramsis less than 0 or greater man 'he height cf target minus tne 
height of model 

• If model is NULL 

• If params is NULL 
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If target is NULL 
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6 Point Registration Tool 
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3 Line Finder 



An Overview of the Line Finder 

The Una Finder ccmputes me iccaticr.s cf lines in an image. For each line that it fines, it 
roturis an ang!« d and a signed distance dfrcm m* canuai otxai cf !fw tmaga. GK/an a and 
d t ycu can compute all feints (x.y) in me image that satisfy tJie equation of itia line (as 
illustrated in tna secuon Tne Transformation from Cartesian Scaca ic Hough Space on 
page 2C4). The algcrimm mat me Line Finaer uses, along wim the angie/aisrance 
classification cf lines, is sxclained in How the Line Finder Wcrks on page 166. 

Ycu can use the Una Finder in either cr ?*c nodes, quick cr normal. In quick mcce tne Line 
rincef srcvices tne tccaucn cf :ne pemt aicng eacn tine tnat is nearest ;o tne center or tne 
image, along with me ar.gie cf me ime. In normal mcce, the Line Finder also estimates the 
lengtn ana aonaity of eacn lira mat it lecansa, using statistical metnocs, and returns mo 
position cf the center of mass ct tne line. 

Figure 73 iilustrates me two modes of ma Line Finder. The image in Figure 78a is snewn m 
Figure 780 after me Line Finder nas seen invoked in quick mode. The presence of the four 
lines is indicated. Each line is drawn to me screen at a constant length, but no attempt ;s 
made to estimate its actual lengtn. Figure 78c shows me resuit when using me Line Finder 
in normai mode: me length of each line segment is estimated. 

For a cetaitec explanation or me caia raturr.ee oy me Line Fmcer in eacn moos, see tne 
section Results in Guide Mcce ana Ncrrv.at Mcae. 




a b c 



Figure tq. An image (a), it nas rcuna tn quicx moae (D), ana t/ne 
segments found in normal moda (c) 

The Line Finder fines tines regardless cf grey scale variations in the image, rotation of the 
scene, or changes in scale. A tyoical acolicatlon tor the Line Rncer is locating fiducial 
marxs in a ser tes cf images in which gray ieveis, scale, and rotation are unoredictaoie from 
ore image to the next. Figure 79a anc Figure 79D illustrate two images. 9acn containing 
the same fiducial mark: a scuare within a square. However, the suss cf the marks, and their 
rotations differ oetween me :wo images. Also, me mark in Figure 79a is a cam square within 
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a lighter square against 3n even lighter oacxgrcund. but r.e mark ;n Figure 79b is a light 
square within a darker square against a darker bacxgrcunc. As sncwn in Figure 79c and 
FtQure 79d. tne Line Finder finds the cudmes ot tne riccc:ai marks cesoite these variations. 




d 



F/gure 79. The Una Finder ;s immune rc grey scate, s/ze ana rotation 
cnanges 

SlncG rna una rincsr :s an interrr>eciais- ; evei -cci. ycur acoiicancn win srccaoiy require 
that you do furtner processing, using tne results oi ms Lne Fincsr as input. You wiil 
probaeiy not be soarcnmg :or linos m ycur image, suit ratnsr for a fgature wncse presence 
is implied by me icc3tlcns ct lines. Per sxampie. in Figure 79, me Line rincer does not 
return tfi© locations and dimeneione cf :ne flcuciai marKa. oniy tne focencna of tne lines tnat 
bound tne nested squares. Computing the creese locations and si2es of the fiducial marks 
requires post-prccassing oased cn ma Line fincer's resuits. 

The Una Finder demo code includes post-processing d this sort. It contains functions thet 
locate squares of any size and rotation, using tne Line Finder. This coce deduces tie 
presence of souares oasad on the Line finder's resuits, eliminating extraneous lines. See 
Chapter 1. Cognex API introduction, :n '.he Qeveiccment Environment 1 manual for me 
location and name of the Contour Finding cemo coce cn ycur system. 
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3 Une Finder 

How the Line Finder Works 



This chapter exclains how trie Une Finder fines lines in an image. This discussion is divided 
into Ti& foi lowing cubcacione: 

• Defining a Line by Angie and Distance aescnoes the definition of a line usee oy the 
Line Finder. 

V Hougn Spaca introduces ma two-cimensicnai soaca in wnicn tne Lne Sneer records 
tines. 

» The Hougn Unv Transform drccriDes tne aigsritnm tnat tne Line finder uses Hnc 

lines In an image. 

The Line Finder uses a Cartesian coordinate system whose origin is Tie center of the image. 
Mos: of the examples in tne cock are drawn in this coordinate system, although a few use 
image coordinates. These rwo systems are otcrured in Figure 30. Nonce mat tne x-axes cf 
the two systems are tne same, but the y-axes are opposite. Note also that positive angles 
in Cartesian coordinates are measured counterclockwise from the x-axis; positive angles 
In image buffer cocrcinates are measured cioexwise from the x-axis. 









y 


Image buffer 




cccrainaies 






v 



Cartesian 
coorcinates 




Figure 30. Image butter coordinates and Cartesian coordinates 

The Line Finder uses :he central pixel of the image as a reference point. The image buffer 
coordinates of this oixel fx* yjare computed with integer division, using the following 
formulas: 
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c d 



Figure 32. The angle of the distance vector determines the sign of the 
disrsncs 

8v dasicnatinc a sign :c ihe distance vector, Eased on its angle (5 In Figure 32c, £+180 in 
Figure 32a), aacn line is ansured a unique definition, in rhis example, tne two lines are 
defined as (8. d) and (6. -d). 

Once you know the distance and the angie of a fine, you can compute all points (x+ y 0 ). in 
Cartesian cccroinates. mat he on me line. Given a distance, .d and an angie 8. a line is ail 
points /x> y 0 ) that satisfy tne following equation: 

The derivation of thia formula ia aupolied in the auction The Transformation from Cz/tecion 
Space to Hough Spaca on page 204. As an example of its acoiicaticn, if 8 is 45 and dis 0, 
a line is ail points in Canesian scacs that satisfy the following equalities: 
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y e - h/2 

wnere Wis me wiatn of me image ana Wis me height cf me image, :n cixels. in me Line 
Finder, me Canesian coordinates of me central pcint in me image are (0,0). 

Defining a Line by Angle and Distance 

The Line Finaer defines a line (in Canesian coordinates) by its angie frcm trie x-axis ana Dy 
the signed distance frcm me center of :he tmage (the central pixel) to the line. These two 
parameters uniquely describe any line in Cartesian scaca. 

The feature in Figure 81 a contains an edge that lies along a line (Figure aib). The tine is 
defined Py its angle from the horizontal (8) and by the sncnest distance from the center of 
me image to me line (Figure Si c). Note mat ;ne distance vector :s perpendicular to me line. 




a S 



Figure a ? . Angis ana atstancs cefine a Una 

In the definition of a tins oy angie ana distance, the cistance is negative if tne angie of the 
distance vector (in Canesian coordinates) is greater than or equal to C° and less than 180°: 
the distance is positive if me distance vector has an angle greater than or equal to 1 80° and 
;iess than 360\ 

Distance is signed to avoid the ambiguity illustrated in Figure 82. Figure 82a shows an 
image wrm two features. £acn feature is ocroered on one side Dy a line. Aitnougn tne lines 
are different, they nave me same angle 6 and me same aoaoiute distance a from the center 
(Figure 82&). 
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0 = xsin(45) - yco$(45) 

0 = x vr y 7. 

' ' [\ * = y 

Hough Space 

Hough space is a rwc -dimensional soacs in which tne Une Finder reccrcs the lines that it 
finds in an image. A point in Hougn spaca represents a line; one axis reoresents angle, the 
other represents distance. 

Hougn spaca is implemented in rne Line Finder as an 3rray; a simple example is shewn in 

— Figure S3. The Hough spaca in mis axampio can racsrd aignt anglec and nine distances 

and 'therefore 72 lines. 

The Line Finder lets you control me size of Hougn 9pace. You specify the range of distance 
in pixels, and. in tne scge detection data structure thai you oass to tne Line Finder, you 
supply the number of angles that tne Une Finder can compute. 



Angle 



315 
270 
225 
180 
135 
90 
45 
0 



-3 -2 



Ftgura 33. Hough spacs 



-1 0 1 
Distance 



The Hough space in Figure 84 contains a single line. It has an angle of 135 a ana is a 
distance cf 3 units from the center ot the image. 
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Angle 



315 
270 
225 
180 
125 
90 
45 



-3-2-10 1 2 3 4 
Distance 

Figure 34. Hcugn space containing a single una 



Imaga 



The Hough Line Transform 



The Line finder uses me Hougn Line Transform to locate fines in an image. This method is 
outlined ana illustrated in tn/s section. To understand mis discussion, you should oe familiar 
with edge detection and ceak detection, as described in Chapter 4, Edge Detection Tcot, 
in tne Image Prccasstng mznual. 

The Hougn Line Transform, as implemented cy tnis :ooi. takes advantage of the following 
rule: If an ecga oaei's angle <* ana location {xO.yO) is known. ;nen tne fine on wnicn the pixel 
lies is also known: ;t is tne line wim angle v*90 that contains the oixel (xO,yG). This is 
tlluetratea in Figure 36. 
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l80 o.:_ ... 

4& ■ 




Figure 55 Four acges in an image (a). 9cge angias at 'cur ecge oixeis 
(b). ana tine angtes at tncse pixels (c) 

The Hough tine Transrorm is implemented as tailcws: 

1 . Uang eage detection, Tie Line Finaer creates an eage magnitude image and an edge 
angie image from the input image. 

2. The Line Finder creates and dears a Hough space. 

3. For each edge pixel in the input image, the Une Finder does the following: 

• Using me ecge angie • ana location (x0//O) or the oixei. 3ie Une Finder caicuiates 
the line's distance from me center. d t and its angte. e (using the formula e = $ * 
90). 

• The Line Finder increments the bin in Houch scaca reoresanting the line (9.d). 

4. Or.ce all ecge pixels in the image nave been examined, the Une Finder searches 
Hougn spacs for maximum values, using peak cstectlcn. The nignest value ;n Mougn 
scace represents the strongest line in me image, me second highest represents the 
ssccna strongest tine, ana so fortn. 

5. The location of each line is primed cut and, optionally, each line is drawn in an output 
image. 

Figure 86 through Figure S9 illustrate this method. 

Figure 86a ;a an tmage passed to the Line Finder. Figure Mb is a stylized edge 
magnitude image created from the input image. 



81 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 D ^rc^ M ^ 

PCT/US97/18268 



3 Line Finder 




Figure £6. An input image (a) and the edge aerec;ec input image (b) 

Each edge pixel :s processed, as described above. For example, because the outlined 
edge o\xe\ in Figure 57 belongs to me fine (90, -2). trie bin in Hcugn space mar represents 
tnar tine is incremented. 



Angle 



315 
270 
225 
180 
135 
90 
45 
-0 



0 


0 


0 


0 j 0 


z 


0 


0 


0 


0 


0 


0 


0 


0 


2 


2 


0 


0 


0 


0 


2 I 0 j 2 


1 


0 


0 


0 


0 


0 


o 1 1 i o 


0 


2 


0 


0 


0 


0 


0 


0 


0 


0 


0 j 2 


0 


c 


o[- 


0 


0 


0 


ol i 


0 


0 


1 


2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


0 


0 


0 


0 


0 




-3-2-10123 
Distance 



Figure 37. The bin in Hough scace for ihe edge pixel's line is 
incremented 
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Rgure 3fl shows tne final Hough space aicng with me input image. Figure 39 shows the 
peak-detecied Hougn space, with aach of :ne peaks outlined. Rgure 69 also snows trie 
incut Imaca with Tie lines, t eor essntsd by the ceaks in Houah soaca. drawn into the incut 
image. 
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Figure 38. The final Hougn soaca ana ma incut image 
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Oisianca 

Figure 89. Trie peax-Cotectcd Mcugn spaco ana mo input Imago, wren 
lines adc&d 

Notice tnartne Hougn space contains many extraneous lines. II. for instance, me vision 
problem is to find the triangle in me imace. the lower scoring lines mat Dcrcer :ne cicb need 
to be eliminated. Almost all cf the Une Finder applications require posx-o recessing cf this 
sort In this axamole, amply locating the tnree hignest oeaks in Hough space eliminates 
the unwanted lines (Rgue 30). In most aoniicaticns. the required cost-orccessing is more 
complex. . 
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Figure SO. The output image after post-processing 
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APPENDIX II 



Patent Application for 



MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION 



Software Listings 



THE FOLLOWING APPENDLX IS NOT BELIEVED TO BE NECESSARY FOR 
ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED AND 
CLAIMED IN THE ACCOMPANYING APPLICATION. 
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Copyright (c) 1996 by Ccgnex Corporation. Natic:<, MA 'J5A 
Ail rights reserved. This material contains unpublished . 
copyrighted work, which includes confident ial and proprietary 
information of Cocnex. 



cva_f ind_targec .c, Aaron Wailack - 13 Apr 1996: Created. 
Time-stamp: <96/07/30 09:32:47 awailack> : last modified. 

♦ 

* SRe vision: /main/ncneS 
♦/ 

^include <prealign.h> 
^include <math.h> 
^include <scdio.h> 
^include <cct.h> 
^include <cip.h> 
"include <otn.h> 
"include <caq/caq_def . h> 
* include <cva_criv.h> 
Stinciude <search/ccr_def . h> 
* include <search/cse_dei . h>. 
S include colp/ccip.h> 
^include <cip/ rcip_er . h> 
* include < chp . 



This file includes a funrc: 
square? target . 



a crosshair 



W *T ^ ^ ^ ^ 



it ^ it it it it it it £ ii i; it it it :t 




The algorithm is intended to handle the cases when the target is 
imperfect (occluded or broken) 
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* These routines are intended to be used as a subroutine for 

* determining the cameras * field of views (with respect to an absolute 

* coordinate system defined by a motion stage) . 

* The target localization routine (cva_find_ target ) takes as input: 

* img - a cip_buf f er containing an image of the target 

* caiib - a calibration structure intended tc characterize the 

* ncn- rectangular! ty of the pixels 

* angle - the orientation of the crosshair in zerr.s of image 

* coordinates (we require this 

orientation estimate to be accurate z z 5 degrees. 2 . : 3 radians 

* resuii - pointer to a rwa_line structure where -he resulc will he 

returned 

* Requirements : 

* Image quality: 

* The image of the crosshair must be visible and perfect 

* (without occlusion cr breaks; throughout a i:o X IOC pixel square 

* surrounding the crosshair center. 

* Consequently, the center of the crosshair must be at least 

* 50 pixels away from the borders of the cic_buffer. 

* Orientation estimate accuracy: 

* The given angle estimate (given m terms cf image coordinates) must be 

* accurate to wiohm 3 degrees C.OS radians 

* This aigoritm tolerates occlusion and breaks in the target 

* so long as these imperfections occur at least 50 

* pixels 'away f rex' the crosshair center 

* Major Consideration; 

* One interesting thing to note about crosshair calibration, and the reason 

* that we use four distinct rays, is that the separation between the opposite 

* rays depends upon the lighting and aperture of the lens. Furthermore, we 

* cannot assume that the separation between these edges will remain constant 

* from one setup to another 

* what we really want is the position of the 

* 333333333*33 
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* since we do net ic.cw what the of f set .* separation is. :: :urr.s ou: that cha 

* repeatability cf the entire fir (Iccalicacicn of the crosshair} is bounded 

* by the accuracy :f the smallest of these fcur lines. It dee sr. • z sake any 

* sense to include 10000 edge points from one ray if we or.!/ have SO edge 

* points from another (the localization error will soil! be or. "he order cf 

* 0 . i/'scrc (50) pels, assuming random i.i.d. errcr and surpixel localization 

* accuracy cf 0.1 pixel) . 

* We war*: to incorporate one sar.s number :: edge pernio from opposite rays 

* because we always wane oo ::: -he lines to the same exact points for 

* highest accuracy and repeatability' . -hen "he calibration target is 

* occluded by a particular object isuch as one borders that can be seen in 

* Che database:- . then if we use one exact same number of pixels on the 

* opposite unocciuded ray as the r.uober of pixels cn the occluded ray, then 

* we will always be locking at the sar.e pixels on the ■unocciuded ray 

* Basically, since we do net know which edge points correspond to 

* imperfections in the target and which edge points correspond to 

* valid acmes cn the target, we use a bect - strapping algorithm to 

* compute regions where edge points are VALID . 

* Assuming that the target is perfect within the GUARAiiTZED GCCD 

* REGICN {center */- SO pixels; . we can compute four GCCD RAVS 

* characterizing the four rays emmatmc from the crosshair center - 

* Then, we can use these GOCD RAYS to define the VALID REGIONS (the 

* sec of points within a TIGHT jn r 3SSKOLO of the GCCD RAYS). Finally, 

* we improve the . estimates of the four rays by . going back and' . 

* enumerating edge points along the GCCD RAYS until we encounter an 

* edge point which fails outside the VALID REGIONS. 

* Definitions 

* GUARANTEED GCCD REGICN 

* region surrounding the origin where we insist that the calibration target 

* must be perfect 

* GCCD RAY 

* A rav fit to points enumerated within the guaranteed good region 

* VALID REGICN 
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Regie- surcunding geed ray where we :r:s: thai -he edge po:r.;: are due ce 
the crosshair and net from occlusion or breaks 

OUTLIER POINTS 

Peine s which we cannot cruse because they dc ncc fall v;:h:r. Che valid 
region 

cz^rrz^ o? crosshair target 

V cuclier 

V edge poincs 

ti^i±^^s#^ < coed ray - > 



VALID REGION 



: < guaranteed geed regicr. 

We are adopting a better- safe- char.- scrry scraoegy fcr handling 
occlusicn (as seen as we see a " bad" pome, we lose faith in all of 
che subsequent poincs) . Although chis approach is subcpcimal. the 
database images seem co contain nearly perfecc poincs emmacmg 
from che origin chac ic should work well in practice. 

Algorithm: 

I; Use abseiuce ncr-malired correlation search co apgrc^ir.acely 
locace che crosshair center 

This involves constructing a syr.checic model cc a rotated 
crosshair cencer (see grcss_locace_cresshair - cencer_ic_image . : . 

2) Given chis miciai pcsicicn escimace and che user -supplied criencacion 
estimate, improve che pcsicionai and oriencacional escimaces by Iccaliring 
four good rays eminacing from che crosshair cencer. This involves 
enumerating edge points wichen the GUARANTEED GOOD REGION (50 pixels) 
along the supplied orientation as well as 

theca+?I/2, checa>?l-> theta+3*?I/I . These computed four GOOD RAYS are used 
in step 3 to check 'Whether edge points are outliers. In other words, these 
four geed rays define VALID REGIONS separating good edge points (points we 
believe correspond co the crosshair) from bad edge poincs (points we 
believe are due to occlusion or breaks) We should also peine out that we 
fix che orientations of the four rays by "averaging" these orientations, 
i.e., we incorporate the constraint chat che four rays correspond to 
orthogonal physical lines. (see 

compute center_positicn_and - angie_assuming_f cur_guaranceed_gocd_rays () ) 

3) Go back and enmerace edge poincs along che four rays until we find an 
edge point which dees ncc satisfy che VALID REGION constraints. We also 
throw out seme of the edge points in order co realize equal numbers of 
edge points from opposite rays. (see 
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compute center_ccsiticn_and_angle_aicng_guaranteed_gccc rays ; ■ 
NOTES - 



* When we enumerate edge pcints along rays eminatmg from che origin . 

* we start out a small di seance away from the origin sc that we don't 

* gee confused by two different edges (this buffer zone is 

* characterized by the constant cr?.CTI_Axl_3Ur 7SjZCNH_A£C;^n:_rH:rrz?.. 

* which is 10 pixels) 

* enumerate_jpoints_and_f it_line_alcr.g_ray ( ) which enumerates edge points 

* along a specified path. enumerate_points_and_£ it_lir*e_aicng_ray { ) applies 

* calipers at sample points along che path (she edge estimates can be 

* further improved using parabolic suhpixel interpolation ; Along a specific 

* path, enumerate_pcints_and_f ic_iir.e_alcng_ray { ) always makes integral unit 

* steps in either the x or y direction. This is sc that ve tan apply 

* axis -aligned calipers along ever*/ row or column when the step size is 1 

* enumerate ocir.ts_and_£ii_lme_alrr.g_ray , enumerates edge points 

* until it .either encounters a bad edge :c::.: ( :r finishes sampling 

* the edge segment. At this point, it ceases er.umeratmr edge :c;r.:: . 

* and fits a line to the points. There are ::ur ways an edge point 

* can be "bad". If er.umerate_pcir.ts_ar.d_f it_lir.e_aicr.g_ray ■ '. 

* encounters any o; these four situations, i" returns ? after fitting 

* the line; if it successfully enumerated all of the edge points 

* along its specified course, i~ returns 1. 

1. no edge is found by the caliper applied a: zhe sample point 

2. mere than one edge is found by the caliper and ihe highest 
scoring edge is less than twice the score of the second highest 
scoring edge (this comes from the CC:i"3ICN_TK?.Z3KC_T, : 

3. the computed edge point is an outlier ! .zhe distance between the 
edge poinc and the nominal line is larger than threshold) 

4. the caliper extends outside of the cip_buffer 

* When we call enumerate_poir.ts_ar.d_f it_line_aiong_ray • ) in' step l, we 

* specify a ray from a point near the crosshair center to a point 50 

* pixels away from the crosshair center. We do lock at the return val 

* to check that the enumerate_ccints_ar.d_£ ii_lme_aiong_ray ( ) did not 

* encounter any problems . 

* When we call enumerate_pcints_and_f it_iine_along_ray : ; in step Z. we 

* specify a ray from a point close ro the crosshair center along a 

* good ray to a point 1000 pixels away (guaranteed to be outside the 

* cip buffer). Thereby, even if we do not see any "bad" points, we 

* will undoubtedly leave the cip_buffsr at seme point. 

* The reason we pass a calibration object to cwa_f ind_target is that we 

* recompute the orientations of the four rays by incorporating the knowledge 
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that they all come from orthogonal lines. If the pixels are r.cr.-square 
(i.e., by virtue of the fact chat the CCD elements are nor. - square . . then 
we need to acraun: far this when we "average" the f cur orientations. We 
"average" the f cur orientations by transforming the image orientations 
into physical coordinates (and then truly averaging the physical angles; 
and then transforming the average angle back into image coordinates . We 
try to use the term "phys" to denote physical coordinates and "img" to 
denote image coordinates (such -as phys_orient and img_trient) . 

We use different sized calipers for steps 2 i 3 because we make 
different assumptions about the relative accuracies of our caliper 
application points. We use the terms LCOSc and TIGHT tc refer to the 
calipers; LCOSZ for step 2 where we have little faith in the 
application points, and are thereby forced to use bread calipers, and 
TIGHT for step 3. where we have much, more faith in the application 
points, and can thereby use relatively tight calipers. 

We expect the initial search position to be accurate zz Z pels, and 
the orientation estimate to be accurate tc C . 3 5 radians . At a caliper 
application point SC pels away, the caliper may be zzz by up tc 5 
pels from the correct edge position. Just tc be sate, use a 
caliper with projection length 2-i '. 12 cr. earn sice: zz enumerate ecge 
points in step 2 

After improving the posititn and crientatitn estimates in step Z. ve 
basically assume that we've get the right position tt - - 0.2 pels 
and 0.01 radians. Thereby, we can 'use these lines tt threshold 

outliers because we vcuid expect errors cf only 2. 2 eels at ICC 
pixels away. This is pretty tight because ve use a threshold of 2 . :• 
pels to detemme outliers 

(actually, the C.I pels. 0.21 radians could significantly impact the 
performance because the first 50 pels and the or lentatitr. estimate 
end up predominantly affecting which edge points are ir.rluded m the 
final line estimates: 

Main subroutines - 

grcss_locace_crcs3hair_center_in_image : ) 

construct a- synthetic model' of t the crosshair and use search ' to 
estimate the crosshair position 

f it_line_tc_comts : : 

fit a line (x.y.t) to an array of points 
ccmpute_subpixe imposition ( ) 

use parabolic interpolation at the neighborhood cf the application point 
to compute the subpixei edge position 

enumerate_pcints_ar.d_f it_iine_aicng_ray ( ) enumerate edge points along a 
line segment from start_pcmt to end_point and possibly store the found 
edge points in an array. Furthermore, a nominal line and a threshold are 
passed in as arguments, and when the found edge points deviate from thes 
nominal line by mere than the threshold, enumeration is stopped and a line 
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* is fir to the enumerated edge positions . Enumeration is also stopped when 

* two edges are found (confusion) , no edges are found (occlusion; . or when 

* the caliper is applied out of bounds (out of bounds. 1 — ~ - r 

* average_f cur_image_orientations 0 _r i . 

* average the orientations of the four rays to compute the best 

* composite orientation estimate. This function transforms the image . 

* orientations into the physical domain in order to exploit the 

* constraint that the physical lines are orthogonal v : y ^ z r 

* cctnpuc eocene er_pcs i ticn_and_angie_assuming_f our_guaranteed_go^qd_rays { ) 

* improve estimate of the crosshair position and orientation by 

* searching four regions of guaranteed good length (SO" pixels) 7 r - 

* eminating from the crosshair center. Lines are fit to the" edge 

* points along the four regions, and then the orientations of these 

* lines are "averaged". Two orthogonal lines are computed by 

* averaging pairs of the four rays Finally, the position estimate is 

* computed by intersecting the two orthogonal lines 

* ccrr.pute_cenrer_rcsi t ion_ar.d_ar.gie_aicng_guarar.teed_gccd_rays ; generate 

* final estimate of crosshair position and orientation by enumerating points 

* along fcur rays (using four good rays as thresholding lines . m We use 

* tighter calipers ! than we used m step 2 . because we trust the positions 

* of the good rays. 
•/ 

^define N_SSGiS 10 - 
^define GUASASTiiS_5CCD_SI-i 30 

^define GRGSSJTARGZT^SirZ GUA.^"£-_GC0D_5:ZZ 
^define NGM_SAMPLZ_?Oi:r:3 1000 
See fine LCCSZ^TERZZKCLS i- . 
Sdefme 7IGKT_TK?.ZSKCLZ 3. 
Sdefins LCCSZJ^Z?Z?.J«ZZZ}i :4 

/* ( (int) 2- iLCC£Z_TKRZSHCL3-GUA^A:rrzzr_GCCC_S:-Z*sinC0. Z5> ) - 2 (kernel}- 

Sdefine 7IGHT_CU.r?EH_WI3TH 14 

/• ( iiacS 3»TIGST_THAi5HCLZ-) - 2 (kernel; v 

/* 

* TIGKT_CrtlI?EA - rtZ3TH should be larger than the minimum width so that 

* we check whether there is a. sharp edge just outside the 

* TIGKTJTHRZ5K0LD rang 2 . because if there is a sharp edge just' 

* outside the TIGKT_TH?-£SKCLD range, then we want to declare 

* confusion and stop enumerating points 
•/ 

^define LCCSZJ^r?sa_PRCJZC7IC*I_:.Z:icrH 5 
^define TIGHT^CVLIPSR^PaCJiCTICN^LSIGTH 5 
^define MAX_MUM_?OINTS 1000 
^define SPSIL0n"tHR2SKOLD 0.05 

^define CZRCUL^_3V?T^_ZCirE^CX^JZS^n^ ( 3 . ^LOOSZ^CAilPSR^PROJSCTICN^LZNGTH) 
^define S7Z?_3IzEj;S2DjrC_S2raiZaA7Z~^ 3 
^define STZ?_SIZZ_aSZD_TC_H2^^HATz2ALl_?CI2n , 5 1 
^define MAX_LZ2IGTH_DIAGCNAL_ACRCS5_C2? - 3u7rSR 1000. 
#def ine CCNFUSICN THRZ5HCLD 3 . 
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/* auncer cf pixels to ignore when we run into an outlier r 

Sdefine 0CTLIsa_3UcrE3 7 

extern int cd_shcwas ( in- . char*) ; 

extern double cva_poi.nc_discar.ee 0 ; 

char* cy_preaiig— _cva_£ ir.d_carget { ) ( return ?ILZ ; 

static int cva_f ind_carget_debug_f lag=0 ; 
inc sec cva find tarce c_debuc ! int x) 

{ 

int o!d_£lag=cva_f ind_cargec_debug_f lag; 
c va_ £ ind_carge c_debug_ £ 1 ac = x ; 
return old_£lag,- 



cypedsf enua calib_c riant 
{ 

} eaiib_orient; 

/* A training parameters record fcr :r = :r.:r.r recatec rrcsshair 
static ccr_paranis cva_calib_ccrr_tp = 

ma:<2 a binary model .2 grey levels 
r.c leniency 
don't measure ancle 



{ 2. 






0. 




/• 


0 , 




/* 


5 . 




/• 


1. 




/♦ 


1 , 






50 , 




/ * 


4, 4 , 


/* 


model 


0 , 




/♦ 


0, 




/ 


o ( 




/• 


o.o, 


/• 


r.c sz 


123, 


/* 


cars 


0 




/♦ 



at 1% 
. at I* 



/♦ (do net bother to initialized reserved fields; 



static cse_params 

cva_calib_sp =* {cse_ahsciuts , cse_abs_binary; NO, NC, 1, ISC , 1000} ; 

/* caliper used to find mere accurate positicn estimates of edge 
• positions 
*/ 



static 

cclp_ccnstraint cva_f ind_targec_gcs [1] 



\ ; 



1C00, 



100. 
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/♦ static •/ 

cclp_caliper cip_find_ targe t_initial_s ear ch = ( 
LCOSE_CALI?ER_VTDTH, /• Search Length ♦/ 
LOOSE JtALI?ER_?RO JECTION_I^NGTH. /♦ Projection Lengt 

2, /* Edge Filter Size 

0, /* Edge Filter Leniency */ 
10., /» Expected Size */ 
EEGE_TKRESKCLD. /* Contrast threshold •/ 
CCL?3oNT_CAHS. /• El Polarity •/ 
C-CLP_NC_?AIRS , /• E2 Polarity • / 
CCLP_DE7AULT, /* Optistizacicn */ 

3, /* 3its per pixel V 
TRUE, /* Window Rotation */ 
TRUE. /* Interpolate */ 

1, /* Number of contraints */ 
cva_f ir*d_target_gcs, /* Constraints ♦/ 
NULL, /* Pixel Map 

0, 0, 0 



TIGKT_CALIPSA_"rf~TH. /• Search length v 
T:G~_CALI?ER_??.CJECTICN_1E:;o~. .-■ - Prefect i: 
2, /* Edge Filter Size 

0, /• Edge filter leniency 
10., /* Expected Size 
E3G2_THR2SKCLD , /* Contrast threshold 
CCLP_CONT_CARE , /• El Polarity 
CCL?_NO_?AlRS , /* El Polarity 

C CLP _DE FAULT, /* Gpcirr.izat ion 

8, /♦ Bits per pixel • 

TRUE. /• Window Rotation 

TRUE. /* Interpolate 

1, /* Number o: contra mt s 
cva_£ ind_target_gcs , / • Constraints 

NULL, /• Pixel Map 

o, o 



}; 



/* static */ 

cclp_caliper clp_f ind_target_ccr:rpute_suhpixel = { 
TXGHT_CALXPER_WIDTH. /* Search Length */ 
!./♦ Projection Length •/ 
2, /• Edge Filter Size 

0, /* Edge Filter Leniency V 

10., /* Expected Size •/ 

EEGE_THRZSKOLD , /* Contrast threshold 

CCL?_DCNT_CARS, /* EI Polarity 

CCL?_NO_ PAIRS, /♦ S2 Polarity 

CCL?~QEFAULT , /* Optimization 

8, /* Bits per pixel v 

FALSE, /♦ Window Rotation •/ 
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FALSE. /* Interpolate 

1, /* Number of ccntraints v 

cwa_find_target_gcs, /* Constraints v 

NULL, /* Pi-xei Map _ _ , _ .*/ . 

0,0.0 " . . I " * 

t. ■ * > * 

/* make an image of a rotated center of , a crosshair . this irtage is 

* used to train a model to search for^ the crosshair in an' image 

* img - cip buffer where rotated crosshair *" will be stored 

» orientaticn_in_tenths - orientation of rotated crosshair in tenths 

* of degrees 

* returns ima * 
*/ 

cip_buffer *make_ercss_hair_cencer_image ^ „ 
(cip_huffer *imc, inr crientaticn_in tenths) 

{ 

cip_bufrer win. *xhair=><ULL ; 
ccc_signai sig; 

i xhair: ; 

if (lime) : . . 

cct_error \CZzZt_Z?S._SAZ^P.C) ; - J- 

if (sig=ccc_catch ; 0 ) ) 

goto dene; _ . 

/* allocate a temporary ci?_huffer {xhair* which will contain a 

* non-rotated crosshairs, and then use this temporary rip_buffer 

* as a source for cic_rccace centered ... 

xhair = cio create ( imc- ^width^t , img- jheignt*Z . 3 ; 
ci?_set {xhair , 0) ; 

cip_window (xhair. iwm. C , 0 , xhair- >width/2 . xhair- > height. 2} ; 
cip_set ( iwin. 255} ; 

cipjwindow (xhair , iwin. xhair - > width/ 2 . xhair - > height /: . 

xhair- >width/2, xhair - >height/2 ) . ... - 
cip_set (iwin. 255)"; 

cip rctate_centered (xhair. img, orientaticn_in_tenths/ ; 
done : 

if (xhair) cip_delete (xhair! , xhair *NCLL; 
if (sig) cct_throw(sig> ; 

return img; 

} 

/* construct a synthetic image and use search to find a gross estimate 

* of the center pes i tier* of the crosshair center 

* we exnect that the synthetic model will be accurate zz . 
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* approximately 1 pixel, and therefore, this search routine r should ... 

* localize the center co approximately 1 pixel .-. < ~ ^ 

* . r-i~ cr -1 . 

* ixg - image containing a rotated crosshair ^ ^ .. " . 

* orier.taticn_in_tenths - orientation of rotated crosshair^ "in~ tenths' 

* of decrees 

* res - pointer to cva_pcint where result will be stored ^ ; 

* gross J.ocatejrrcsshair_center_in_image C returns _int_ sig^i^ij.g ^ 

* whether or not it found a crosshair , ^ T . . ^ T "1^/^ - * 

static int gross_lccate_crcsshair_center_in_image 

(cip buffer *img, int orientation in_tenths. cva jc:r.: ;:es ) 

cip_buffer •targetstfULi.; „ , . 

cse_model ♦!tdl=NULL; 
cct_signai sig; 
cse_results cse_res; 
int ans; 

N0_R£GIS7Z?. : target ) ; 
N0_RZGIS7I~ ;.T.dl; ; 

if (sig=cct_catr:; .'3; !- , - 

goto dene; 

target = c i?_cr eat a !G?.C S3 _7A£GI7_3 "Z . 3?.CSS_7A?.GZ7_S 1 2^ . 3 * ; _ ^ 

make_crcss_::air_cent5r_:.mace target . crier.tatJLcr-_in_ter.ths : ; - 

mdl = ctr_tram_T.cdel : target. Z. Z Z. Z. :. icva_calib_ccrr_tp C- 

cse_area_search:img, mdl. icva_c3lib_3p . icse_res: ; _ 
if (cva_fir.d_target_decug_f lac i 
crir.tf("*d *c \d Vd\n'\ 

cse_res .x. cse_res . y. cse_res . score . cse_res . fcund; ; 

res->x = l. • {G£0S3_7AF.GZ7_5IZi/: - CIAJINEIS i cse_res . x . n: ' ; 
res->y = 1 . • <GRCS5jrA3G27_5I22/ : - CIAJUIDIS :cse_res . y .r.: ; ; 
done : 

"if (target)' cip_delete (target) , target > MUU,; '.'■•» 
if (mdl) • ctr_deiete_mcdei (mdl) , mdl = 
if (sig) cct_thrcw(sig) ; 

return cse res. found; 

} 



/* compute distance from a point to the closest point on a line */ 
stat' c double distance_f rcm_coint_to_line (cwa_point *pl . cwa_l: 

( 

cwa_Iine 12.- 
cva_ooinc 

12.x = pl->x; 
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12. y = pl->y; 
12. t = 1I->C * PI/2; 
cwa_lines_tc_pcint (ll . &12 , &p2) ; 
return (sqrt ( (pi- >x-p2 .x) * (pl->x-p2 .x) * 
(pi- >y-p2 .y) * (pl->v-p2 ./))); 
} ' ' 

/* Fit a line to an array of points */ 

static double f it_ Iine_to_ points 

(cwa_pciat *pts, ist n ccincs r cva_line *line) 

{ 

double X3um=0 . 0 , ysun=0.0, xcent, ycent; 
dcubie a. b, c, theta, xp, yp ,- 
c va_p c in t *p t r ; 

if (n_pcir.cs < 2) 

ccc_errcr (C*A_S^ JfC_INTXRS£C7J ; 

far ( j =0 . per = pes; j <o_pc:.r.ts ; 3 — , ?:r — 
if (cva_find_target_debuc_f lac i 0x1: 

print f V:\n" , per- >x . ptr- >y: ; 
xsua *= pcr->x; 
ysum + = c:r->v; 

i 

xcer.t = xsum / appoints; 
yesnt 3 ysum / n_pcints; 

for (j=0, per = pts, a=o=c = 2.; j<n_ points; : — , ?;r- 
xp = ptr->x - xcent; 
yp = ptr->y - ycent; 
a += xp * xp ; 
b 2 * xp * yp ; . 
c VC ♦ */o ; 

) 

theta = atan2(b, a-c) / 2.0; 

line->x = xcent;-. 
line- >y = ycent; " 
line->t 3 theta; . 

if (cva_f ind_target_debug_flag & 0x1) { 
printf ( , *fit_iine_to — points returned: Vf*\ 
(a*oos ( theta-?I/2 ) *ccs ( theta^?I/2; * 
b-cos (theta-?I/2) -sin ( theta+?I/2) - 
c*sin(theta-*?I/2) *sin ( theta-*PI/2) ; /n_paints! 
cd showas ( (int) line. "cva line 1 *) ; 

} 

return ( (a*ccs (theta^?I/2) •cos ( theta-?I/z: - 
b*cos (tneta+PI/2; *sin ( theta+PI/2) - 
c*sin( theta+PI/2) *sin : theca-r?I,'2) ) /n ooints'; ; 
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static int print_iist_of_points 
(cwa_coint *vai. int num_vais) 
{ 

inn i; 

for ( i = 0 ; i < num_vais; i — ) 

printf <"*f *f\n-. val[ij .x,vai{i] .y) ; 

} 

/* calih_crient_frcm_angle 0 returns CALI3_vz™ or CALI3_rZCRIZ 

* depending upcn the orientation (measured in degrees) of a line. Its 

* purpose is to determine whether we need to use a vertical cr 

* horizontal quadratic suhpixei edge interpolation 
*/ 

static calib_orient caiib_crient_f rem angle (double ancle m_degreesi 

{ 

double angie_in_rad; 

angie_in_rac = angie_m_degrees * 130 . ; 

if (fabs (cos ;ar.gis_ir._rad. > 

£ abs ( s in ( ang 1 e _ ir._ r ad ; 

return J/Z" ; 

return HC?,IZ, 



computes subpixel edge pcsiticr.s given an image and a neighborhood 
(ap_x, apj/) to search for an edge. Depending upcn the 
orientaticn_in_degrees . ccmpuce_subpixei_pcsicicr. i ) will enher 
perform a vertical cr horizontal suhpi;cel edge estimation. Subpixel 
edge positions are computed in ihe same manner as the boundary 
tracker (using 7 neighboring pixels, and 7 differences where 
we quadra ti tally interpolate the edge position from at least : 
differences. 



We compute a seven first differences lust "o be safe. There is no 
guarantee that the computed caliper position corresponds to the maximum 
1st difference (because we're using calipers with filter sice 2} 



* img - cip_buffer containing image of crosshair target 

* a ?_*.ap_y " application' point where we sample image -grey values' 

* actually, we sample grey values ac the application 

* point and +/- 2 from the application point (maybe we 

* should sample at 4. who knows 

* oriencation_in_degrees - orientation of caliper which has been applied 

* co find this application point. Since we 

* only want to call subpixel interpolation 

* at the right place, we suggest using 

* cn-axis calipers 

* pos x. ?os_y - pointers to doubles where 

ccmpute_subpi;cei_pcsition;; will store result 

*/ 

int compute_subpixel_position 
(cip_buf f er *img, int ap_x. int ap_y, 

double orientation_in_degrees . double *pcs_x. double *ccs_y) 
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ir.c p0,pl.?2,?3. p4 . p5 , p6\ p? ; 
inc d0.dI,d2.d3,d4.d5.dS. djnax. 
calib orier.c direccicrt; 



c mm ; 



if (cva_f ind_cargec_debug_f lag i 0x1 ; 

princf ( "ccmpuce_subpixsl_pcs:.c:.c:: called with Vd \d \: 
ap_x, ap_y , crier.rac:.cr:_i:i_decrees . pcs_x. pcs_y ; 



if (ap_x < 4 | i 

ap_x > irrtg- >width-4 | ! 
ap_y < 4 | ! 

ap_y > img - >heighc - 4 ) { -» - 

re cum 0 ; 

} 

/• should we sample he nzcncally cr vercmally 
direction - calib orient f rem ancle i cnencacm 



» /. — 



p0= * f imc- >rac [ ap_y-4 ) -imc- >x effcec-ap :•: 
pi = * ( imc - >rac ; ap_y- 3 ] ~img- >x_cf f sec -ap_x 
p:= r {imc- ?ra: is.p_y-2) -imc- >x_cf f 3ec-ap_::: 
p3 = * { irr.g orac ap_y - 1 • - mc - >x of f sec-ap_x. 
p4= * { imc - >rac [ap_y] -img- >:<_cf f sec -ap_:<; ; 
p5 = * ( ;r.c - >ra: 'ac_y-l] - imc - >x_ef f sec-ap_x. 
p6 = * ( :t.c - >rac [ap_y-Zi ~imc- >::_of f sec -ap_x; 



p 7 = • ( imc ->rac ( ap_ 

dO^pi-pO ; 
dl=p2-pl; 
d2=p3-p2 ; 
d3=p4 -p3 ; 
d4=p5 -p4 ; 
dS=sp6-p5 ; 
d6 =D7 -oo ; 



y-3 ; *img- >x offset *ao 



d_max = max ; do . max £ dl , max t d2 , max ;' d3 , max { d4 . max ; dS . d6 ; * 
d_mia = min i dO , mir* ( dl , mm I d2 , mir. ; d3 , mir. i d4 , min ( d5 . dS ) ) 

if {-d_min > d_rna;<:) { 
dO=-dO; 

dl=-di; 
d2=-d2; 
d3^-d3; 
d4*-d4; 
d5=-dS; 
d6=-d6 ; 

d max = -d mm; 



*pos_x = ap_x* 1 . - 0.5; 

/* If we're ac a maximum re cum che sub nix el position 
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(d max > E£GE_TKjtESK0L3i { v . ; \ 

if (dl == d_max ii d3 ! = d_max ) { 

*pos_y a ap_y «■ cz_parf ic ;dC . dl . dZ) .'5 553S . -I . ; 

recum L; ' , . 

} else if (dl == d_max &« d4 : = d_maxi ( 

*? cs _/ 3 a ?_y * cz^arfic *dl.d2. d3) /5553o. : 

recurs 1; 

} else if (d3 as d_max dS != d_max; { ,>., ■ 
-pes y = ac y > c: oarfi- (d2 , d2 ,d4) .-5HS36 . ; ? 

} else if (d4 « d_raax &i d6 ! = djnax) { glr^^^t 
*pos_y a ap_y > cz jarfit (d3 . d4 /dS) /S5536 - ? *1 . ; 

} else if (d5 as d_max) ( -> ~~ 

*pcs_y = ac_y + cz_parf it (d4 , d5 , dS: /6 553S . *2 . ; 



} 

/* If we're net ac a maximum. 



recurz 3 scmechir 



else 


( 












p0 = * 




>rac 


;ap_y: 


- i.T.c - 




-ap_x--; : * 


pi = * 


Umg- 


>rac 


[ap_y! 


- irr.g - 


>x_crrsec 


-ap_x- 3 ; 


?2 = * 


£ img- 


>rac 


i*ap_y! 


-img- 


>x_sf f sec 


-ap_x-:; ; 


p3 = * 


[ l.T.C- 


>rac 


iap_y! 


- rrng - 


>x_cf f sec 


-ap_x- 1. ; 


p4 = * 


i img- 


>rac 




* img - 


>x_cr f sec 


-ap_x: ; 


p5 = * 




>rac 


[ap_yj 


- img - 


>x~cffsec 


~ap_x-l! ; 


pS = • 


imc- 


>rac 


Up y} 


* i.T.g - 


>x effsac 


-ac_x-«: 


p7 = * 






[as y! 


- img - 


>x_sf r sec 


-ap_x-3 ' ; 



dO=pl-pO; 
dl-c2-pl; 
d2=p3-p2; 
d3=p4 -p3 ; 
d4=pS-p4; 
dS=p6-p5; 
d6=p7 -pb ; 

d_max = maxfdC ,max (dl , max Jd2, max *d3 . max id4 ,max (dS ,d6) ) J ) ! ) 
"d mia = min (dQ> mia ;dl . mr.n id2 , min (d3 . min id4 , mih idS , d5\) K!< ) ; 



{-d_min > d_tnaxi { 
dO=-dO; 
dl=-dl; 
d2=-d2; 
d3=-d3; 
d4=-d4; 
d5=-d5; 
dfi»-d6, 

d max =-d min; 



/* If we're ac a maximum, ccmpuce che subpixe^ 



rosizicn and 
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** return i . » . 

•/ 

*?os_y = ap_y*l. * 0.5; 

if (djnax > ZDGZJT^RZS'dOLD) { 

if (dl == djnax &i d3 ! = d_maa) { 

*pcs_x = ap_x*i. + cc_parf it idO.di. d2: /6553o . -2 . ; 
return 1 ; 

} - 

else if (d2 m d_max d4 ! = djnaoc) { 

♦pos_x = ap_x*l. * cz_parfic(dl, d2, d3) /5 = 53o. -1. ; 
re cum I ; 

) 

else if (d3 == d_ma.x dS != d_max. 1 { 

*pcs_x = ap_x-i . * cr_parf it idZ , c3 . d4 » /S533o . ; 
re cum 1; 

) 

else if ( dl - = d_ma,x dS ! = d max: * 

*pcs_x = a p _ :c * 1 . - cc carf ;: 'dl , dl . dS '. 5 5 52? . -1. .- 



/ 

else if ids == d_.*nax: { 

-pcs_x = ap_x-l. - c=_parfi-:c-i.d3.zo./-5 = 535..:. ; 

} r " 

/* If we're rx: a: a rr.axirr.urr.. scrr.echir.g ' s wrcr.r, and rev-rr. 3 
•/ - 

re cum 0; 



enumerace^cir.cs_and_f ic_I ir.e_almg_rav ; ) enumeraces edge points 
along a ray frcm a scar:j:c;r.: to an endjpcir.t by using calipers 
(and possibly subpixel interpolation; to ccmpuce edge positions 
enumerace_points_and_f ic_iine_aiong_ray ( i continues enumerating 
points • until one of the following conditions occurs: 

0. we reach the end__coint 

* 1 . no edge is found by the caliper 

* 2. more than one edge is found by the caliper and the highest 

* scoring edge is less than twice the score of the second highest 

* scoring edge (this 2X comes f rem rhe CCN~S I0N_7K3E5KOL3 ) 

* 3. the computed edge point is an outlier (Che distance between the 

* edge point and the nominal line is larger than threshold) 

* 4. we run off the cip_buffer (screen! 

* After it has enumerated the edge points, ic then calls 

* fit_poincs_t online to compute the cptirnai line estimate 

* img - cip_buffer containing a crosshair target 



102 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 t>^™_ 

PCT/US97/18268 



start_ccint. end_ccir.t - defies region along whirr, tc apply calipers 
nominal line, threshold - used tc determine whether ar. edge pcmc 

is an outlier (and consequently end the 
enumeration of edge points). The outlier 
test involves checking whether the 
distance from the point to the 
step_sice - frequency of edge point sampling. Since we want to 

evenly sample the lines, we move ste?_sire number of 
pels in the larger direction (and a fractional number 
of pels in the other direction) v This approach provides 
even sampling assuming we are doing on -axes calipering 
or parabolic sub pixel interpolation 
search_caliper - caliper used to search for edge points 
ccmpuced_iine - cva_line where result will be returned 
enumerated_ccints - optional array of cwa_poincs where found edge 

pcints would be stored . . . 
num_ points - pointer to an int where the number of ftund pcints 
would be stcred 

do subpixei_interp - flag characterizing whether we want tc per ft—. 

subpixel interpolator, after 3::.y:r.r :r.s calir 



static int enumerate^ pcincs_and_f ic_ime_alcr.g_ray 
(cip_cuffer 'img, rvajc::.; »scart_pomt. cwa^pcmt *er.c_pc 
cwa_lir.e *ncminal_line , double threshold, ccuble step_3ire. 
ccip_caiiper ♦searrh_caiiper . cva_lme *cor.puted_line , rwa_pomt 
♦enumerated^ocmts, int T.unjc;:.:: . int do subpixel_ir.terp 

{ 

int i. 3 ,3c.- 

double maxAbsCosSin; 

cwa_pcinc ap_pcint. •enum - pomt . vec; 
int ap_x, ac_y ; 

ccip_results res_h IM_ETGE3 * 1 - 1 ] ; 

colp_params cpp = { ZZGZ ^-'—Zl-Z'-Z * 1 j . N'_ZZ3E3. 2): 

double cclp^angie; 

in t num_samp 1 e s . tmp ; 

double caiiper_verr ; 

ccc_signal sig; 

cwa_line sampiinc^line ; 

if {c-^a_find_target_debug - flag i O.xl) { 

princf { w enumerated - point3_and_fit_iine_aicng_ray called with n" ;- ; 

printffimg Vx start_point Vf \i end_point *f *f\n" , 

img, start_pcint - >x. 3tarc_point- >y, end^point - >x. end — point - >y) ; 
printf C 1 ncminai_line Vf V: t \z threshold *f \n" . 

ncminal_line->x. nominal line - >y, ncminai_lme- >t . thresholds ; 

} 

/* Compute the sampling line */ 
sampling_line.x = start_pcir.t - >x; 
sampling_line. y = scart^pcmt- >y ; 

samplmg_line . t = atan2 (er.d_ccmt- >y- start_coir.c - >y. 

end jpo int - >x- star t_cc int - >x: ; 
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/* Compute ~be seep which is integral ir. x cr y :ccn:r.a:e 

if (f abs (cos (3ampiing_ime . ci i > f abs (sin (sampling.- me . z. 

staxAfcsCosSin = fairs ices ( samp l:r.g_ line . z: * ; 
else 

maxAbsCcsSin = f ais (sin (sampiing.line . c) ; ; 

vec.x » (cos (sampiing_line . z) /xaxAbsCcsSin ; * scep_size; 
vec.y =- (sin(sampilng_line. t) ,'maxAbsCcsSin) * scep_si=e; 

/* Ccmcute the number cf samples which we will be making 
•/ 

num_sainples = 

maxfabsf (int) ! start_ccinc- >x - end_pcmt- >:<) : . 

abs( (int) (start_pcinc->y - er.d_pcir.c- >y? : 

step.size ; 

if ( ! enufflerac2C_3C inc 3 j 
er.ume r a : ad j;c :r. : 3 = 

( cwa cc ir. t * ) chp _al 1 cc a ; r.u~_3 ar.p les-sizec: ; rva - " " 

if ( : num_ccint3 ; 
num_pcints - itrrp ; 

cclp.ar.gle = sampling, line . - • 130 . -9C . ; 

if (do_subpixei_ir-terpi \ 

if {calib orient £rcm_ar.gle ccip_a-gie. = = CA1 13 _* .=.?.- - 

cclp.angie = 0; 
else ccic angle = 90; 

} 

if (cwa find target_debug_f lac i Oxl; 
prmtf 1 "ccip.angie Vf \n" , ccip.ancie; ; 



/♦ Inner Lccp: Enumerate edge pcint 



for (i = 0, ap_point.x = scart_point - >x, ap_ccmt.y = 
start rcint->y, enum_pcint = enumerated_ocint3 , 
♦Tium coincs 3 0 ; 
i < nuzn — samples ; 

avjpcir.z .x * = vec.x. ap_pcir.t.y vec.y. er.'iijc::.: - 
( *numj:ci^w3 ) *- ) { 

ap_x = tint) (apjcinc.x * 0.5); 
ap_y = (int) iap_ccint.y - 0.5); 

if (cwa find targe t_debug_f lag i Oxl'; 

prist f 1 "ap_x \d ap_y *d\n" . ap_x. ap_y: ; 
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resji[0] .found = res_h [ 1] found : ^0; :^^*^r - c ~- - • 

. V. , '*'". : '" W , ' ; ^ , 'V"* 

/• cip_£ransf3rst throws CGZN_iHS_3ADAaG . C:?_ERH_?HLA23R . 

• CCLP S2H_FI7 when caliper region .extends " past the hounds of 

♦ the cip_buffer. We wane co break of f the" search at" this 

• point anyways, so we catch the error and jump to fitting a 

* line Co the points r "* * stu t + 1 : rr 
./ . * : ■ ■n : "r.^5n,iV_v. 

if (sig = ccc_caech(CGZN_saa_3ADARGi )-v;^i- . 
goto fit^line; 

if (sig = ccsj=acch(CI?_saa_PSLADDR) ) — - * 
goto fic_line; ' " ' 

if (sig = cct_catch iCCL?_EKH_Fi?J I 

goto f it_line; . . ^ . • . 

cci?_apply 

{ sear ch_tal;. per . i.r.c.ac^x. ac_y . "Ic^ar.rle , i-tc. res_r. 
cct_end(CCl?_rR?-_. r ZT: ; 

cct_end(Cr?_£^x_?H:U^:3X; ; . - ~ ■ - 7 — • 

cct_end (C3Z:r_r^_3A2ARG:- : ' * ' - 

/* check if edge is r.cc f cund 

if { !res_h[Ol .found} . ^'•fr:^"-" *. ' 

goto fit_lir.e; ~ 

/♦ check if -ore than cr.e edge 13 fcur.d ZCM~Sir:: * 
if (res_h[l; - found > ~-* - 

res_h[0l -score < CCN~31c::jn-:ssSKCL3*res_h Hi . score: { 
if (cwa find_ targe t_debuc flag & 0x1. ( 
cd_shcwas i res_h. '•cclp^resuits" ) ; 
cd_showas ( ires_h { 1 ] , "ccio results"; ,- 

} 

goes fi;_line; 

} 

if (cva_f ind_cargec_derug_f lag & : Oxi) { ; 
princf { "ap_x \d ap_y *d angle Vf pes Vf \n" , 

ap_x, ap_y, ccip_angie, res_h[0] .position) ; 
if (fabs {rss_h(Q] .position) > !.){ 
cd_showas ( res_h, " colour esul ts rt ) ; 
cd showas (ires_h[l] , ,, ccio_resuits" ■ ; 

} 

) 

/♦ compute the suopixel point measured by the caliper * ' 

enum_point- >x = 

ap_x * 0.S +■ res_h [0] . pcsition*cos icoip_angie*?I. 1301 ; 
enum ooint->y = 
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ap_y - C.5 * res_h[0] . pest tier.* sir. * tclc_3r.gie*?:. 13: ; 

Sif 0 

if (do_subpixel_interp) { 

if ( : comput2_3ubpixel_ocsiticr. 

( img . ( mt ) enum_cc inc - > x . * inc ; er.unijjo - > y . 
cclp_angie , &enu;n_poinc - >x. i<jr.L-njo;r.: - > y • 
goco fit_line; 
if (cva_find_ targe t_debuc_f lag i 3x1? 

printf ( "cctnpuce subpixel position :s:ur?.ed hi Vf '.n". 
enum ooint- >x, enum_pcint - >y) ; 

} 

*endif 

/* check if che point is within the ACCI?7A3L£ 
♦ (within distance threshold of the r.cminal_iir.e 
•/ 

if (distar.ce_f rtT._cci"t_tc_lir.e • er.ur.^pcir.t . r.crr.ir.al_lir_e: > 
threshold) 



/* fit a line to the points arc return : i i we er.rtur.terec 
* anv bad return 1 otr.erv.se 



f i c_l ine_tc_ccints 

(enu*T.er3ted_ooio.t3 . *r.ur.^c ir.is. rc:ncut£c_lir.e : ; 
} ~" 

static ir.t round nu: roer ! double 
( 

if (x > 0) 
else 

return ; int ; ( x - 0 . 5 ) ; 

} 

/* average two orientations but take into actount chat orientations 
* mod PI ars equivalent 

static double averace cvo_orientat ions "'double orient 1. double oner 

{ 

double dif f , res : 
inc tncdDiff; 

if ( cwa_f ind_ targe c_debug_ flag a 0x1! 
printf ( "avg Vf %f , crient_l , orient^: ) ; 

diff a orient_2 - oriental; 

modDiff » round_nustber (dif f ."?Z\ ; 

res= {'orient l^arient 2-mod£iff .Z) ; 
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if icva_fir.d_target_debug_f lag U :-.i: 

prir.cf ( " « %f\a-. res) ; 
re cum res; 

} 

V* corrcute angle in alternative coordinates giver, by the r.ar 
*/ 

double trar.sf crci_ar.gle_accordir.g_co_calibracicn_:7:ap 
(priv transform »mac , dcuble i.^g_orier. cation: 

{ 

double reaiSin, realCos ; 

- c=_transf omPoin: (map , cos i i-g_criencacicn; , sir. ( img_criencaticn) . 

&rea!Cos , ireaiSin) ; 
" recum(atar.l (reaiSin. realCos) J ; 

>i * - \ 

rf: m /* average the orientations cf the fcur rays tc compute the besc 

* composite orientation estimate. Giver, the orientations z : the fcur 

* image rays ( "he rays should be orient m multiples of ?Z. 2 cue 
» with each other: , compute the "average" crier.caticr. . ~:.r3t. che 

* orientations are aligned img_cner.t ; Z ] sc if ;-c_:r lent ; : ; 

* characterized a hrrircr.cal line, ther. all z : che angles r.ay be 

* shifted by ?!."- tc become her icon tal. tnen we average the resulting 

* orientations. 

* This function transforms tne image 

* orientations into the physical domain m order tc exploit the 

* constraint that the physical lines are orthogonal 
*/ 

static double averace_f cur_image_crier.cacicr.o 

(cva calib *caiib, double *:.t.c orient: 

( 

double phys_orient ] . avgchyscrier.t . avg_img_oriem ; 

* First, transform angles from ir.age coordiantes to physical 

* coordinates 
*/ 

. "' for ti * 0;: i < 4 ; i~i 

phys_6rient [r] = 

trans fcrm_angie_acccrding_co_calibracion_map 
( ( (privjrva_caiib •} calib/ ->map. img_orieric [i] : ; 

if {cwa_£ind_targec_debug_:lag i 0x1: 
for (i * 1; i' < 4 ; i— • 

printf i*\£\n n \ phys_orienc !i; 1 ; 

/• Shift orcbegenai angles by PZ.'Z ir. order to line up with the 
— * first orientation 

•/ 

for ii « 1; i < 4; i-~; 

if (fabs ( sir. (pnys_oner.t [i] -chys_crient -;o: " : >scrc . : 
phys_orient [i) ~~ ?1/1 ; 
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/ • average the orier.:a::cr.s * 
avg_phys_crient = 

average_cwc_orien rations 

(avsrace_cwc_orienca::cr.s : phys_erienc [0] . phys orient ; 
a ve r ag e_ : wc_o r i en : a : : ens ■ phys_crient [2] . phys orient * 3 ; ; ; 

/* compute "lie orientation in image coordinates correspond-no: co 
* che averaged orientation 
./ " 

avg_img_orient = 

transfcrm_angie_acccr:i_ng_cc_caiibraticnjTiap 

( ( ( p r i v_cwa_cal ib * ) cal ih ) - > inv_map , a vg_p hys orient?; 

if (cwa_f ind_target_debug_f lag * 0x1) 

printffphys \z irr.g tf \n" . avg_phys_orier.t. avg i-g_crienc) ; 

return avg_img_cner.c ; 



/* compute che image crisnza:::- ::: which che physical criir.za:;:.- 
* is orthogonal :c another image ;r:er.:3:icr. 
*/ 

static double image_nermal_-e_imace_2ngle 
( cwa_caiib *caiib. icubi e imc orient 

( 



return 

( Crar.sicr77_angle_acccrding_cc_cal1brat1er.jnap 
£ [ • priv_eva_calib v ealib - >mv_map, 
( cransf errr._angis_ac ccrc^nc_to_eal ib racier, jnap 
( Mpnv_ewa_calir * calib. - >map. i-g_crier.c; - ?Z. 



* ccmpuce_cent2r^csizicr._ar.d_ar.gie_assuning_;ciir_gxiaranteed_gcod_rays .* ) 

* improve escimace of che crosshair pesicion and criencacicn by 

* searching four regions of guaranteed good length {SO pixels) 

* emina ting. from che. crosshair center. Lines are fic to the edge 

* points along the four regions "and then the .orientations of these' 

* lines are "averaged-". Two orthogonal lines are computed by 

* averaging pairs of the four rays Finally, the position estimate is 

* computed by intersecting the eve orthogonal lines 

* img - cip_huffer containing an image of a crosshair target 

* caiib - cwa_ calib object characterising pixel ncn- rectangular ity 

* escimated_ctr_i:ng - estimated position (given in image 

* coordinates) of crosshair target (computed in step 1) 

* escimated_orientatien_phys - user- supplied orientation estimate 

* (given in physical coordinates) 

* guarancaed_gecd_discar.es - length (ir* pels) of guarateec good 

* region extending outward from the origin 

* ccmpuced_center - structure where computed position will be stored 
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ccmputed^orier.cacicn^phys - double where tempered :r:er.:a::cr. ir. 

physical coordinaces i will be scored 
good rays - array of four cva_line scruccures where good rays 

~ (lines fit zz sampled edge pomes m the gccd region, 

will be scored 

/ 

static int compute j:enter_pos it icn_and_angie_assuming_f our_guarar.teed_gcod_rays 
(cip buffer »img, cva_calib »caiib. cva_point •estimated_rtr_img. double 
estimaced_oriencation_phys. double num_guaranteed_gcod _poir.cs, 
cwa _point *ccmputed_center. double *ccmpuced_oriencacion_j:hys , 
cwa line *gocd_rays) 

{ 

iac i; 

double angle [4] ; 

cwa_point scart_point [4 ] . end jo::.: [ 4 ] ; 
cva line compuced_line [4 ) . ncminai_lme ; 
double img_angie[4] ,angie_hcric_Iir.e: 



angle [Oj 


= escimaced^tri 


e r. c a c i c r._p hy s ; 


angled! 


= esc:.:nac3d_ori 


encarior._pr.ys -PI. 2 ; 


angle [2] 


= es::T.a:ed_:r: 


er.caticr._phys-?I . 


angle [3] 







/ * define che 5 C pel GuARAN*TZZC GCCC xZGICN ir. cerr.s - - a 

* VALID R£GICN so chat we ran use 

* che enumeratej?oincs_and_f ic_line_aicng_ray ■ ; fur.ccicr. 

* If we're sampling along a line and we wane co scop sampling 

* after 50 pels, Chen we can use a nominal line which is ncrr.al 

* co and passes through che scare pome, and a thr es.ee. n zz 
•/ 

nommaljine.x = esti:r.aced_rcr_L:?.g- >x ; 
nominal line . y = escimaced_ccr_L:r.g- >y ; 



/* The passed criencacicn is giver 



:ccrdir.aces 



for (I = 0; i < 4; I-*? 

angle [i] = crans:orm_angle_acccrdinc_co_caiibracion_i\ap 
( ( (priv_cwa_calib *>calibi - >inv_map , angle f .i; ) ; 

for (i = 0; i < 4 ; { 

/♦ The sampling region begins ac che peine a small distance 

* away from che crosshair center 

* (CIRCUIJVR_3t^r£^_ZCNE_ARCL 1 ND_r£NTSR; co be exact, and 

* extends out to guaranceed_gcod_di stance from che center 

•/ 

start_j3oint [i] -x = estimated_ctr_img- >x - 

C^CUI^_3Urr£R_2CNE_AROUND_CZNTZ?.*cos(angle[i] ■ ; 

start_point [i] -y - estimated_ctr_img- >y * 

CIRCtnJUl - 3UFFrR_ZCIIE_AR0UJID_CENTHR*sin{angie[i] ) ; 
end_point [i] -x = estimatedjztr^img- >x ♦ 

~num_guaranteed_gccd_points*cos (angle [i] ; ; 
end ooinc[i].y =» estimaced_ctr_img- >y - 
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scacic int intersect_line_ar.d_cirtle ■ cwa_lir.e -Line, rwaj::;-: -p- . 

double radius, cvajjoir.: *res_:. 
cwajjo ir.t 'res 2 ! 

( 

double angle_co_iine. distance_tc_Iir.e. angle_to_ccir.t_rr._li.-e ; 

angle to line » line- >t*?I/2 ; 
if ( (cos (angie_to_line) * (line- >x-pt->x) * 
sin(angle_to_iine) * { line- >y-pt - >y) ) < 0) 
angle_to_line PI; 
distance to_line = distance_f rcm_ point_to_line (pt , line) ; 

if (distance_to_line > (radius - E ? S I LGN_TKR£S HO LD ) ■ 
c ct_err or (CWA_ERR_N0_ INTERSECT) ; 

angle_to_p< 

res_l->x = pt->x * 

radius *cos (angle_co_line~ar.gle_tc_ocint_cn_line ) ; 
res_l->y = ?t->y * 

radiuses in ;ar.gie_tc_lir.s^ar.gle_~c_pcir.i_::r._lir.e 1 ; 

res_2->x = pc->x * 

radius*ccs ;angie_to_line-ar.gle_zc _pcir.t_cr-_lir-e ; ; 

res 2->y = pc - >y * 

radius*sin;angle_tc_lir.e-ar.gie_io_pcir.z_cr._line: ; 



/* compute_centsr - oosit_cr._and_ar.gie_along_guaranteed - gccd_rays • 

* generate final estimate of crosshair position and orientation by 

* enumerating points along four rays (using four good rays as 

* thresholding lines) . We use tighter calipers because we trust the 

* positions of the good rays. 

* img - cip buffer containing an image of a crosshair target 

* caiib - cwa_caiib object character icing pixel non-rectar.gulanty 

* estimated ctr img - estimated position (given in image 

* ~ coordinates) of crosshair target (computed in 

* step 2) 

* gocd_rays - array of four cwa_line structures where good rays 

* (lines fit to sampled edge points in • the good region) 

* will be stored 

* tighc_threshold * threshold used for determining outlier points 

with respect to the good rays 

* computed center - structure where computed position will be stored 

* ccmputed_prientation_phys - double where computed orientation (in 

* " physical coordinates) will be stored 

*/ 

static int ccmpute^center^position^and^angle^alcng^guarantaed^good^rays 
(cio buffer *img, cva_calib *calib, cva joint *escimated_rtr_img , 

cwa~iine *gocd_rays. double tight_threshold. cva_point 

♦computed center, double *ccmputed_orientaticn_phys , 

double *feature score) 
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if { (cos (orientation_away_f rcm_center) * 

(good rays J i] - x-escimated_ctr_img- >x. - 
sin (orient3tion_away_f rom_cencer : * 
(gcod_rays i i] . y-escimated_ctr_img- >y. < C; 
oriencation_away_£ rcm_center -= PI; 

end_point[i] .x start_paint [i] .x + 

MAX LZ?IG7K_DIAG0MAL_Ara0SS_C:? - BCcriR* 
cos (orientation_away_f rcra_center) ; 

end^jooinc [i] .y = start_point (il . y * 

~HAJC_LZNG73_D IAG0NAL_AC3QSS_CI ?_BUFrER * 
sin (orientaticn_away_f rom_center ) ; 

if (cwa_find_target_debug_£lag & OxI) { 

printf ( "center Vf Vf good_ray \f *£ Vf distance 

estimated_ctr_img- >x, estimatad_ctr_i:ng- >y . 

gocd_rays(il . x, gocd_rays -i ; . y , gccc_rays ! i ! . t. 

distance_tc_center) ; 
printf:"%£ if V: *f'n" . 

s:a::j:::r.: ■ i ; . x, start_pcir.t [l] y 

snd^ pc ; i ] . x . er.d_pc ir.t [ i ! . */ ; 

} 



enunterata_point3_and_f i t_l ir.e_alor.g_ ray 

(img, istart_pcint [L] . iend_pcir.t I i ] . igccc_rays . i . , 
tight_thr eshold . 

STS? - SI2i_ r JSZD_TO_E2TCMEaATS_AI.L_?CINTS. 
_cip_£ind_target_f ine_search. 
/*&cip_fmd targec_ccmput2_subpixel , * 
^computed 1 ire [ i] . &er.umerated_pcir.t3 ; i] ;0 ; , 
inum_er.umerated_poir.t3 ' il , 1; : 
img_angle[ii - ccmouted_lir.e { i] . t ; 

} 

/♦ For each oair cf opposite rays, use the closest points so that 

* we use the same number of points on both sides of the 

* crosshair center 

* Since we don't know why the last edge point became invalid, we 

* skip the neighboring 2 points just to be cn the safe side 
*/ 

num_enumerated_points (0] = num_enumeratsd_points [2 ) = 

min(num_enumerated_points [0] , num_enumerated_points [2\ \ -CXrTLI2a_3UFF2R; 
num enumerated^coints [li = num_enumeratec_pcir.ts [ 3 ] = 

min (num enumerated_points [ 1 ] , num - enumerated_points [ 3 I *• -CUTL.IEa_3t7rTia ; 

/ * Re- fit lines to these points */ 

avg_squared_error = 0 ; 

for (i a 0; i < 4; i — ) { 
double fit error ; 
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cwa ocizic gross oos escimace, improved_pos_esci;7iate .- 
double improved ohys_orientacion_escimace .- 
cva_line gocd_rays (4 ] ; 
inc i . an 3 ; 
ccc_signal sig; 
ccm_cimer cime; 
inc CO, tl, C2, C3; 
cva_line c:ro; 

<rwa_calib *calib_idencity=NULL; 
N0_aEGISTZ2{calib_identicy> ; 
cu_ciear (scacus, sizeof (cwa_scacus) ) ; 

/ * CWA_SRH_NO_ INTERSECTION is used to signify when ve rarr.cc f ind 

* any edge points for an expected horizontal or vertical line 
•/ 

if v 'sig = ccc_catch;0: .■ 
goto dene; 

,'* e:cpand the stack co make rrcrr. izr tr.e icuble arrays w-e 

* allocate 

cu_expand_ stack ( IS I ; 

if ( ! result I 
result = itmp; 

if ! : imc) 

cct_error ;C3E2I_SHK_3ADA?.G- ; 

/* If no calibration object is passed, r.ake one so that there is 

* only one path through the code 
*/ 

if (icaiib) 

calib = calib identic*/ = cwa_calib_:nake : 0 . , 0 . , 0 . , 1 . . 1 . , 0) ; 
/* 1) use search.*/ 
ctm_begin ( icime ) ; 

ans = gross_locace_crosshair_cencer_in_image 

(img, (int) (10 . * angle*190/?Ii , igrcss_pcs_estimate. ; 
if ( : ans ) crt_error (0*A_ERR_NC_ INTERSECT J ; 

{ 

cip_buffer win; 
double x_edge. y_edge; 
inc edge_val , dir ; 
double angle_of f sec; 
ccc_signai sig; 
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iresult- >x. ^result- >y) ; 

if (cva_f ind_target_debug_f lag £ Oxl 5 ( 
princf ( "result :") ; 

cd_showas ( ( ) result, "cvajjcir.c"! • 

} 

if (cva_f ind_target_debug_flag £ 0x4) { 
printf ("Times;") ; 
princfr gross locate: Vd\n ,# . CO) 
princf(" fine locate: %d\n" . tl- tO) ; 
printf ( " finer locate: td\n M , t2 - tl) 

} 

done : 

cu_expand_stack { - 16 ) ; 

if (calib_identity) cva_caiib_delete icalib identirv; ; 

/* Make returned angle agree with given angle argument v 

if (sig) cct_tr.row(sig) ; 

for { 1 = -9; i <= 3; i--} 
if ( (i != 0) 

(fabs (result ->t - i*?l. : - angle: < fabs . result - >t - angle 
result - > t «■= i *?!.' 2 , i - - . 

return 1; 

} 

int cva_f ind_target 

(cip_buffer » ixg , cva_calib *calib. double ancle. cwa_line -icsul;, 
cva_status * status) 

{ 

cva_status t:np_status ,- 
cct_signal sig; 

if (: status) status = it^tp_status ; 

cct_catch(CGEN_ERR_BADARG) ; 
cct_catch(CTA_£Rii_BACKLIT) ; 
CCt_catch(CTA_ERR_TRACXHR) ; 

if (sig=cct_catch(0) ) { 

if (sig =x= CGEN_ERR_BADARG) { 

status- >error_f lags i = CWA_ERR_3ADARG ; 
cct_end (sig) ; 
cct_error (sig) ; 

} 

status- >errcr_f lags | = CWA_i££_FAILZD_7C_r ZlID JTARGT? ; 
return status- > found » 0; 

} 

cwa_f ind_rargec_i^camai (img, caiib, angle, rasulc, scacus) ; 
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Described herein are calibration targets and methods for determining the location 
and orientation thereof meeting the objects set forth above. It will be appreciated that the 
embodiments shown in the drawings and discussed above are merely illustrative of the 
invention, and that other embodiments incorporating changes therein fall within the scope 
5 of the invention, of which we claim: 
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6. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a Hough line vision tool to the image 
and finding an intersection of lines identified thereby. 

5 7. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a correlation vision tool to the image 
using a template substantially approximating an expected pattern of edges in the 
image. 

10 8. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a projection vision tool to the image 
along one or more axes with which the edges are expected to be aligned. 

9. A method according to claim 1, comprising the steps of 

15 

identifying edges in the image corresponding to lines in the calibration target; and 

determining an orientation of the target based on the angle of those identified 
edges. 

20 

10. A machine vision according to claim 1, wherein 

step (B) includes the step of applying a Hough line vision tool to identify edges in 
the image corresponding to lines in the calibration target; and 

25 

the method further including the step of determining an orientation of the target 
based on the angle of those identified edges. 

11. A machine vision according to claim 1, comprising 

30 

A. applying a Sobel edge tool to the image to generate at least a Sobel angle image; 
and 
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step (C) includes taking an angle histogram of the Sobel angle image to determine 
an orientation of the target. 

15. A method according to any of claims 1 and 12, wherein step (A) includes the step 
5 of generating an image of a target in which each region has any of a different 

color, contrast, brightness and stippling from regions adjacent thereto. 

16. A method according to claim 15, for use with a machine vision system having 
image capture means for capturing an image of the target, wherein each region has 

10 a different characteristic in such an image than the regions adjacent thereto. 

17. A method according to any of claims 1 and 12, wherein step (A) includes the step 
of generating an image of a target for which the reference point is at a center 
thereof 

15 

18. A method according to claim 17, wherein step (A) includes the step of generating 
an image of a target for which the linear edges substantially meet at the reference 
point. 

20 19. A method according to any of claim 1 and 12, wherein step (A) includes the step of 
generating in image of a target having four imageable regions. 

20. A method according to claim 19, wherein step (A) includes the step of generating 
an image of a four-region target in which the at least two linear edges of each of 

25 the regions are perpendicular to one another. 

21. A machine vision method of determining a location and orientation of a calibration 
target in an image thereof, the method comprising the steps of 

30 A. generating an image of a target that includes 
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i. applying a Hough vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 



5 ii. applying a projection tool vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 

Hi. applying a correlation vision tool to the image to find approximate 
10 coordinates of the reference point; and 

iv. determining a sum of absolute differences to find approximate 
coordinates of the reference point. 



15 24. A method according to claim 23, wherein step (D) includes the step of invoking 
the steps in sequence, at least one time: 



i. applying a caliper vision tool along each of the expected edges in the 

image, where those edges are defined by a current estimate of the 
reference point location and a current estimate of the target orientation; 
and 



ii. fitting lines to edge points determined by the caliper vision tool. 

25 25. A method according to claim 24, wherein step (D)(i) includes the step of applying 
the caliper vision tool along each of the expected edges until any of the following 
conditions is met: no edge is found by the caliper vision tool at a sample point; 
more than one edge is found by the caliper vision tool at a sample point; a distance 
between a sample edge point and a nominal line is larger than a threshold; or, the 

30 caliper vision tool window for the sample edge point extends outside of the image. 
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